Calligraphy and painting appreciation interaction method and device based on eye movement capture

Abstract

This invention discloses a method and device for interactive calligraphy and painting appreciation based on eye-tracking, relating to the field of virtual reality technology. The method includes: on a backend server, using an improved pigeon flocking optimization algorithm to optimize parameters, configuring and deploying the parameters of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction; on an eye-tracking digital terminal, collecting real-time eye-tracking interaction data; mapping the real-time raw gaze point sequence to the texture coordinate system of the calligraphy and painting digital model, and filtering it using a moving average filter to obtain a real-time smooth gaze point sequence; based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence, using an interest evaluator to assess interest and determine the interaction lock area and its cumulative duration; generating an interaction trigger command based on the cumulative duration, sending it to the backend server, and executing calligraphy and painting appreciation auxiliary behavior. This invention achieves a highly immersive, anti-interference, low-latency, and highly interactive calligraphy and painting appreciation experience.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to an interactive method and apparatus for appreciating calligraphy and painting based on eye-tracking. Background Technology

[0002] With the rapid development of digital museums, virtual reality (VR), and augmented reality (AR) technologies, traditional methods of appreciating calligraphy and paintings are evolving towards digitalization and interactivity. Eye-tracking technology, as a non-invasive, intuitive, and natural interaction method, can accurately reflect the distribution of users' visual attention and cognitive interests, and is gradually being introduced into calligraphy and painting appreciation scenarios.

[0003] However, existing interactive systems for appreciating calligraphy and painting based on eye-tracking suffer from the following technical bottlenecks in practical applications: 1) High noise in eye-tracking data leads to positioning drift: Due to the limitations of the hardware precision of eye-tracking devices and the user's slight head movements, the original gaze point sequence collected usually has high-frequency jitter. If it is directly mapped to the texture of a high-resolution digital model of calligraphy and painting, it will lead to inaccurate judgment of interactive hot zones.

[0004] 2) Weak generalization ability of interest assessment models: Traditional interest assessment often uses fixed fixation duration thresholds or simple pupil diameter changes to judge, which cannot adapt to the physiological differences of different user groups and the visual complexity of different calligraphy and painting exhibits, and is prone to high false alarm or false negative rates.

[0005] 3) Algorithm parameter configuration relies on human experience: Key parameters such as filter window size, duration weight, and pupil weight in the system are mostly set manually through repeated debugging, which is not only time-consuming and labor-intensive, but also makes it difficult to find the globally optimal parameter combination, resulting in a poor interactive experience after the model is actually deployed.

[0006] 4) Limited computing power and interaction latency of the terminal: If a complex deep learning interest evaluation model is run directly on the eye-tracking digital terminal, it will result in excessive power consumption and serious heat generation of the terminal; on the other hand, if all the raw data is uploaded to the backend server for processing, it will cause serious network latency, which cannot meet the real-time interactive requirements of "what you see is what you get" in calligraphy and painting appreciation.

[0007] Therefore, there is an urgent need for an interactive method for appreciating calligraphy and painting based on eye-tracking that can adaptively optimize parameters, operate lightweightly at the edge, and has high anti-interference capabilities. Summary of the Invention

[0008] This invention provides an interactive method and device for appreciating calligraphy and painting based on eye-tracking. This invention solves the problems of high noise in existing eye-tracking data, reliance on human experience for interest assessment parameter configuration, high false alarm rate, and difficulty in balancing edge computing power and real-time performance.

[0009] In a first aspect, embodiments of the present invention provide an interactive method for appreciating calligraphy and painting based on eye-tracking capture, the method comprising: On the backend server, the improved pigeon flock optimization algorithm is used to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal. The eye-tracking digital terminal receives real-time eye-tracking interaction data of the user with respect to the calligraphy and painting exhibits, which is collected by the eye-tracking device. The real-time eye-tracking interaction data includes real-time raw fixation point sequence and real-time pupil diameter sequence. The real-time raw gaze point sequence is mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and the obtained real-time mapped gaze point sequence is filtered using a moving average filter to obtain the user's real-time smooth gaze point sequence. Based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence, an interest evaluator is used to assess interest, and based on the obtained real-time interest score, the user's interaction lock area for the calligraphy and painting exhibits and its cumulative duration are determined. Based on the cumulative duration, the system generates an interaction trigger command for the user's interaction-locked area, sends it to the backend server, and executes the corresponding calligraphy and painting appreciation assistance behavior through the backend server.

[0010] The technical solution provided in this application has at least the following beneficial effects: By using an improved pigeon flocking optimization algorithm for offline parameter optimization on the backend server, the complex optimization process is separated from the terminal. Only a lightweight interest evaluation function based on simple weighting and threshold judgment is deployed on the eye-tracking digital terminal, greatly reducing the computational burden on the terminal and ensuring millisecond-level real-time response in eye-tracking interaction. A moving average filter with dynamically decaying weight kernels is introduced to effectively filter out high-frequency jitter noise from the eye-tracking hardware, making the gaze trajectory mapped onto the digital model of calligraphy and painting smoother and more accurate, improving the robustness of hotspot determination. For the specific scenario of calligraphy and painting appreciation, an innovative fitness function that integrates prediction accuracy and false alarm rate is constructed. Simultaneously, the traditional pigeon flocking optimization algorithm is improved by introducing Logistic chaotic mapping initialization to enhance global search capabilities, and a map compass operator with adaptive convergence factors and a landmark operator with weighted centers are introduced to improve convergence speed. The long-tail characteristic of Cauchy mutations gives the algorithm the ability to escape local optima (suboptimal parameter combinations), thereby automatically finding the most suitable filtering and evaluation parameters for the current user group and exhibit characteristics. In interest evaluation, in addition to focusing on traditional gaze duration, it also creatively introduces the pupil relative change rate feature with "micro-fluctuation dead zone threshold" and "pupil extreme value reference upper limit", effectively eliminating the interference of micro-fluctuations in pupil caused by changes in ambient light, preventing normalization anomalies caused by physiological limit expansion, and improving the accuracy of appreciation intention recognition from a physiological perspective. The interactive trigger command only contains extremely low payload data such as timestamp, ID and hash value. Combined with a three-stage anti-shake state machine and a background anti-replay verification mechanism, it not only saves network bandwidth, but also completely avoids the phenomenon of accidental triggering when the user's gaze sweeps over, ensuring the accuracy and elegance of the timing of appreciation assistance behaviors (such as audio and video push, knowledge graph pop-up).

[0011] In one optional implementation, an improved pigeon flock optimization algorithm is used on the backend server to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy it to the eye-tracking digital terminal, including: On the backend server, historical eye-tracking interaction data of several users with calligraphy and painting exhibits is collected. The historical eye-tracking interaction data includes historical original fixation point sequences, historical pupil diameter sequences, and real interaction intent labels. Extract the static feature vector of each historical eye-tracking interaction data and add the static feature vector to the corresponding historical eye-tracking interaction data to construct a training dataset; The parameter vector consisting of the moving average time window, gaze duration weight, and pupil dilation weight is encoded into the position vector of an individual in the improved pigeon flock optimization algorithm. Construct a fitness function, and based on the training dataset, use the fitness function and an improved pigeon flock optimization algorithm to optimize the parameters and obtain the optimal parameter vector; Based on the optimal moving average time window of the optimal parameter vector, configure the moving average filter for calligraphy and painting appreciation interaction, and based on the optimal gaze duration weight and the optimal pupil dilation weight, configure the interest evaluation function for calligraphy and painting appreciation interaction. Deploy the moving average filter and interest evaluator to all eye-tracking digital terminals connected to the backend server.

[0012] In one alternative implementation, a fitness function is constructed, and based on the training dataset, an improved pigeon flock optimization algorithm is used to optimize the parameters, resulting in the optimal parameter vector, including: Construct a fitness function, generate a chaotic sequence using a Logistic map, and map the chaotic sequence to the parameter space of individuals in the improved pigeon flock optimization algorithm to obtain the initial population. The initial population is updated using the map compass operator to obtain an updated population. Based on the training dataset, the fitness function is used to calculate the fitness value of each updated individual in the population in one update, and the individual with the best fitness value in one update is updated as the best individual. Based on the fitness value, the bottom half of the individuals in the updated population are eliminated, and the center position of the remaining population is calculated. Based on the center position, the remaining population is updated to obtain a second-updated population; Based on the training dataset, the fitness function is used to calculate the fitness value of each individual in the population that is updated twice, and the individual with the best fitness value is updated as the best individual. Perform Cauchy mutation on the best individual to generate a trial individual, and add the trial individual to the population updated in the second iteration as the population to be updated in the next iteration; The position of the population is repeatedly updated. When the number of iterations reaches the maximum number of iterations or the fitness value of the best individual meets the requirements, the iterative optimization of the population is terminated, the position vector of the best individual is output, and the position vector of the best individual is decoded to obtain the optimal parameter vector.

[0013] In one optional implementation, the static feature vector includes the original average pupil change rate and the actual effective fixation frame number of any consecutive eye movement segment falling within the local hot zone of the calligraphy or painting in the training dataset.

[0014] In one alternative implementation, the fitness function is formulated as follows: In the formula, For individuals XThe fitness values ​​of the corresponding alternative parameter vectors in the training dataset; For individuals X The prediction accuracy and false alarm rate of the corresponding alternative parameter vectors on the training dataset; This refers to the fitness weighting coefficient.

[0015] In one optional implementation, a moving average filter for calligraphy and painting appreciation interaction is configured based on the optimal moving average time window of the optimal parameter vector, and an interest evaluator for calligraphy and painting appreciation interaction is configured based on the optimal gaze duration weight and the optimal pupil dilation weight, including: Convert the optimal moving average time window of the optimal parameter vector into the number of sampling points; Configure the number of sampling points to the size of the filter window of the moving average filter in the calligraphy and painting appreciation interaction. Configure an interest evaluator for calligraphy and painting appreciation interaction based on the optimal gaze duration weight and the optimal pupil dilation weight.

[0016] In one optional implementation, the real-time raw gaze point sequence is mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and a moving average filter is used to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence, including: The coordinates of each real-time original gaze point in the real-time original gaze point sequence are mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, resulting in a real-time mapped gaze point sequence with mapped coordinates. Using a moving average filter, extract the filter window preceding the current time step from the real-time mapped gaze point sequence. Real-time mapped gaze point sequence segments; Calculate the smoothed coordinates of each real-time mapped gaze point in the real-time mapped gaze point sequence segment to obtain the user's real-time smoothed gaze point sequence.

[0017] In one optional implementation, an interest evaluator is used to assess interest based on a real-time smoothed gaze point sequence and a real-time pupil diameter sequence. Based on the obtained real-time interest score, the user's interaction lock-in area for the calligraphy and painting exhibits and its cumulative duration are determined, including: Based on the real-time smoothed fixation point sequence, extract the real-time pupil diameter sequence segment from the real-time pupil diameter sequence; Based on the ray method, it is determined whether the smoothed coordinates of each real-time smoothed gaze point in the real-time smoothed gaze point sequence fall within any hot zone. If so, the cumulative duration of the gaze hot zone is counted. Based on the real-time pupil diameter sequence segment and the cumulative duration of the fixation hot zone, the real-time relative pupil change rate is calculated, and based on the real-time relative pupil change rate, the interest assessment tool is used to calculate the real-time interest score of the fixation hot zone. If the real-time interest score is greater than the interest score threshold, the corresponding gaze hotspot will be used as the interaction lock area, and the user's interaction lock area for the calligraphy and painting exhibits and its cumulative duration will be output.

[0018] In one optional implementation, based on the cumulative duration, an interaction trigger command for the user's interaction-locked area is generated, sent to the backend server, and the backend server executes the calligraphy and painting appreciation assistance behavior corresponding to the interaction trigger command, including: Determine whether the cumulative duration of the interaction lock area is greater than the gaze threshold. If so, when the three-segment anti-shake state machine meets the conditions, use the eye-tracking digital terminal to encapsulate and generate an interaction trigger instruction with extremely low load. The interaction trigger instruction includes a timestamp, user ID, artwork ID, and area hash value. The interaction trigger command is sent to the backend server through a pre-set private TCP / UDP port within the local area network; On the backend server, the timeliness of the interactive trigger command is checked, as well as the debouncing and anti-replay checks. If the checks pass, proceed to the next step. The system analyzes interactive trigger commands, uses the local high-performance GPU mounted on the backend server to execute at least one calligraphy and painting appreciation assistance behavior, generates corresponding data packets, and sends the data packets to the eye-tracking digital terminal. The calligraphy and painting appreciation assistance behavior includes backend spatial audio mixing and distribution, backend dynamic image real-time synthesis and push, and backend collaborative knowledge graph rapid retrieval. Using an eye-tracking digital terminal, the local rendering engine is invoked to render and present data packets, and the latest eye-tracking interaction data of the user is continuously monitored to return the gaze point filtering steps.

[0019] Secondly, embodiments of the present invention provide a calligraphy and painting appreciation interaction device based on eye-tracking, used to implement a calligraphy and painting appreciation interaction method, the device comprising: The parameter deployment unit is used on the backend server to optimize parameters using an improved pigeon flock optimization algorithm, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal. An eye-tracking unit is used to receive real-time eye-tracking interaction data of users with respect to calligraphy and painting exhibits collected by an eye-tracking device in an eye-tracking digital terminal. The real-time eye-tracking interaction data includes a real-time raw fixation point sequence and a real-time pupil diameter sequence. The gaze point filtering unit is used to map the real-time raw gaze point sequence to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and to use a moving average filter to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence. The interactive locking unit is used to evaluate interest based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence using an interest evaluator, and to determine the user's interactive locking area and its cumulative duration for the calligraphy and painting exhibits based on the obtained real-time interest score. The interaction trigger unit is used to generate an interaction trigger command for the user's interaction lock area based on the cumulative duration, send it to the backend server, and execute the calligraphy and painting appreciation auxiliary behavior corresponding to the interaction trigger command through the backend server.

[0020] A third aspect of this invention provides an electronic device, which includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor can perform the method proposed in the first aspect of the present invention.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect of the present invention. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the steps of an interactive method for appreciating calligraphy and painting based on eye-tracking, provided by an embodiment of the present invention. Figure 3 This is a schematic diagram of the functional units of an interactive device for appreciating calligraphy and painting based on eye-tracking, provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0024] The present invention will be further described below with reference to the accompanying drawings.

[0025] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0026] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0027] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0028] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and an electronic program for an interactive device for appreciating calligraphy and painting based on eye-tracking.

[0029] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device. The electronic device calls the electronic program of the calligraphy and painting appreciation interaction device based on eye tracking stored in the memory 1005 through the processor 1001, and executes the calligraphy and painting appreciation interaction method based on eye tracking provided in the embodiment of the present invention.

[0030] Reference Figure 2 The present invention provides an interactive method for appreciating calligraphy and painting based on eye-tracking, the method comprising: S201: On the backend server, the improved pigeon flock optimization algorithm is used to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy it to the eye-tracking digital terminal. S202: In the eye-tracking digital terminal, real-time eye-tracking interaction data of the user on the calligraphy and painting exhibits is received by the eye-tracking device. The real-time eye-tracking interaction data includes a real-time original gaze point sequence and a real-time pupil diameter sequence. S203: Map the real-time raw gaze point sequence to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and use a moving average filter to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence. S204: Based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence, use the interest evaluator to evaluate the interest, and determine the user's interaction lock area and its cumulative duration for the calligraphy and painting exhibits based on the obtained real-time interest score. S205: Based on the cumulative duration, generate an interaction trigger command for the user's interaction-locked area, send it to the backend server, and execute the calligraphy and painting appreciation assistance behavior corresponding to the interaction trigger command through the backend server.

[0031] The technical solution provided in this application has at least the following beneficial effects: By using an improved pigeon flocking optimization algorithm for offline parameter optimization on the backend server, the complex optimization process is separated from the terminal. Only a lightweight interest evaluation function based on simple weighting and threshold judgment is deployed on the eye-tracking digital terminal, greatly reducing the computational burden on the terminal and ensuring millisecond-level real-time response in eye-tracking interaction. A moving average filter with dynamically decaying weight kernels is introduced to effectively filter out high-frequency jitter noise from the eye-tracking hardware, making the gaze trajectory mapped onto the digital model of calligraphy and painting smoother and more accurate, improving the robustness of hotspot determination. For the specific scenario of calligraphy and painting appreciation, an innovative fitness function that integrates prediction accuracy and false alarm rate is constructed. Simultaneously, the traditional pigeon flocking optimization algorithm is improved by introducing Logistic chaotic mapping initialization to enhance global search capabilities, and a map compass operator with adaptive convergence factors and a landmark operator with weighted centers are introduced to improve convergence speed. The long-tail characteristic of Cauchy mutations gives the algorithm the ability to escape local optima (suboptimal parameter combinations), thereby automatically finding the most suitable filtering and evaluation parameters for the current user group and exhibit characteristics. In interest evaluation, in addition to focusing on traditional gaze duration, it also creatively introduces the pupil relative change rate feature with "micro-fluctuation dead zone threshold" and "pupil extreme value reference upper limit", effectively eliminating the interference of micro-fluctuations in pupil caused by changes in ambient light, preventing normalization anomalies caused by physiological limit expansion, and improving the accuracy of appreciation intention recognition from a physiological perspective. The interactive trigger command only contains extremely low payload data such as timestamp, ID and hash value. Combined with a three-stage anti-shake state machine and a background anti-replay verification mechanism, it not only saves network bandwidth, but also completely avoids the phenomenon of accidental triggering when the user's gaze sweeps over, ensuring the accuracy and elegance of the timing of appreciation assistance behaviors (such as audio and video push, knowledge graph pop-up).

[0032] In one optional implementation, an improved pigeon flock optimization algorithm is used on the backend server to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy it to the eye-tracking digital terminal, including: S2011: On the background server, collect historical eye-tracking interaction data of several users with the calligraphy and painting exhibits. The historical eye-tracking interaction data includes historical original fixation point sequences, historical pupil diameter sequences, and real interaction intent labels. S2012: Extract the static feature vector of each historical eye-tracking interaction data and add the static feature vector to the corresponding historical eye-tracking interaction data to construct the training dataset; In this embodiment, the historical eye-tracking interaction data not only includes basic time-series data, but also environmental context data (such as the illuminance of the exhibition hall at that time, which is used for subsequent calculation of the environmental adjustment coefficient). The specific process for extracting static feature vectors is as follows: A sliding window is used to extract segments from the historical original gaze point sequence. Combined with pre-defined local hot zones (such as inscription areas, main scenic areas, and blank areas) in the digital model of calligraphy and painting, eye-tracking segments that "continuously fall within the hot zones" are selected. For each segment, two core features are calculated: 1) Raw mean pupillary change rate: reflects the peak level of physiological arousal of the user within this segment; 2) Actual effective fixation frames: The number of pure fixation frames after removing frames lost due to blinking eye movement data, reflecting the true fixation intention. These two features are concatenated with the true interaction intention label (obtained through user post-event questionnaires or button confirmation) to construct a high-dimensional training dataset. S2013: Encode the parameter vector consisting of the moving average time window, fixation duration weight, and pupil dilation weight into the position vector of an individual in the improved pigeon flock optimization algorithm. S2014: Construct a fitness function, and based on the training dataset, use the fitness function and an improved pigeon flock optimization algorithm to optimize the parameters and obtain the optimal parameter vector; S2015: Configure the moving average filter for calligraphy and painting appreciation interaction based on the optimal moving average time window of the optimal parameter vector, and configure the interest evaluation device for calligraphy and painting appreciation interaction based on the optimal gaze duration weight and the optimal pupil dilation weight. S2016: Deploy the moving average filter and interest evaluator to all eye-tracking digital terminals connected to the backend server.

[0033] This step is the core foundation of the entire interaction method, aiming to solve the problem of poor generalization ability of fixed-parameter models caused by the different visual complexities of different calligraphy and painting exhibits and the large differences in eye-movement physiological characteristics among different user groups. It adopts an architectural paradigm of "offline retraining / optimization in the cloud and lightweight online inference on the device."

[0034] Specifically, the backend server can extract historical eye-tracking interaction data from the database periodically (e.g., weekly) or for newly added specific calligraphy and painting exhibits. Because calligraphy and painting appreciation exhibits a distinct "visual wandering-lingering and detailed observation" characteristic, even small changes in parameters can lead to drastic fluctuations in interest determination results. Therefore, an improved pigeon-swarm optimization algorithm with powerful global search capabilities is employed. After optimization, the backend server packages the final parameters into JSON or binary configuration files via over-the-air download technology or a centralized distribution mechanism over a local area network, and pushes them to all eye-tracking digital terminals within the exhibition hall (such as VR headsets and naked-eye 3D large screens with accompanying eye-tracking camera processing hosts), achieving silent hot updates of the terminal models.

[0035] In one alternative implementation, a fitness function is constructed, and based on the training dataset, an improved pigeon flock optimization algorithm is used to optimize the parameters, resulting in the optimal parameter vector, including: S20141: Construct a fitness function, use a Logistic mapping to generate a chaotic sequence, and map the chaotic sequence to the parameter space of individuals in the improved pigeon flock optimization algorithm to obtain the initial population; The formula is: In the formula, For the first n+ 1. n There are several chaotic variables whose values ​​range from [0, 1]. The stability coefficient is typically 4. This sequence is ergodic and random, ensuring that the initial population is uniformly distributed in the solution space, avoiding getting trapped in local optima, which is superior to traditional random initialization. n Indicator of chaotic variables; In the formula, For the initial population, the first i An initial individual; For the first i One chaotic variable; These are the upper and lower limits of the parameter space; i For individual indicators; S20142: Based on the training dataset, use the fitness function to calculate the fitness value of each initial individual in the initial population, and take the initial individual with the best fitness value as the optimal individual. S20143: Perform a map compass operator update on the initial population to obtain an updated population, using the following formula: In the formula, For the first t+ 1 ,t In the first update of the population in the second iteration, the first... i In a single update, during the initial iteration, For the initial individual; For the first t+ In the population updated in one iteration, the first i The speed at which an individual is updated; t This represents the current iteration number; In the formula, For the first t In the first update of the population in the second iteration, the first... i The speed at which an individual is updated; For the first t The optimal individual in the next iteration; For the first t The convergence factor of the next iteration; For compass factors, take a random number vector within (0,1); In the formula, This is the threshold for the number of iterations; e The base is ; S20144: Based on the training dataset, using the fitness function, calculate the fitness value of each updated individual in the population in one update, and update the individual with the best fitness value in one update as the best individual; S20145: Based on fitness values, eliminate the bottom half of the individuals in the updated population and calculate the center position of the remaining population using the following formula: In the formula, For the first t+ The center position of the remaining population after one iteration; For the first t+ The remaining population in the first iteration i The remaining individuals; For the first t+ The remaining population in the first iteration i The fitness values ​​of the remaining individuals; N The total number of individuals in the population at one update; S20146: Based on the center position, update the remaining population positions to obtain a second-updated population, using the following formula: In the formula, For the first t+ In the population of the second update in the first iteration, the first i The speed of the second update for individuals; S20147: Based on the training dataset, use the fitness function to calculate the fitness value of each individual in the population that is updated twice, and update the individual with the best fitness value to the best individual. S20148: Perform Cauchy mutation on the optimal individual to generate a trial individual, and add the trial individual to the population of the second update as the population for the next iteration. The formula is: In the formula, For the first t+ A trial individual in the first iteration; For the first t+ The optimal individual in one iteration; The scaling factor for variation is set to 0.05 and decreases linearly with iteration. To generate a random number vector that follows the standard Cauchy distribution, the algorithm utilizes the long tail characteristic of the Cauchy distribution, which is flatter than the normal distribution, to give it the ability to jump out of local optima (such as suboptimal parameter combinations) with large steps. S20149: Repeatedly update the position of the population. When the number of iterations reaches the maximum number of iterations or the fitness value of the best individual meets the requirements, terminate the iterative optimization of the population, output the position vector of the best individual, and decode the position vector of the best individual to obtain the optimal parameter vector.

[0036] In one optional implementation, the static feature vector includes the original average pupil change rate and the actual effective fixation frame number of any consecutive eye movement segment falling within the local hot zone of the calligraphy or painting in the training dataset.

[0037] In one alternative implementation, the fitness function is formulated as follows: In the formula, For individuals X The fitness values ​​of the corresponding alternative parameter vectors in the training dataset; For individuals X The prediction accuracy and false alarm rate of the corresponding alternative parameter vectors on the training dataset; For fitness weighting coefficients; In the formula, For individuals X The corresponding alternative parameter vectors for eye-tracking segments s The prediction accuracy; This represents the actual number of effective gaze frames. This represents the original average pupillary change rate; The gaze threshold; The sampling period; The threshold for interest score; To smooth out the equivalent effective duration; This is the environmental regulation coefficient; Used as the base time window; The function is for finding the minimum value; The lightweight interest score is obtained by the lightweight interest evaluation function of the interest evaluator. This serves as a reference value for duration normalization. For individuals X The corresponding alternative parameter vectors include the moving average time window, fixation duration weight, and pupil dilation weight; s It provides an eye-tracking segment indicator; reducing computational complexity from a large number of operations involving nested temporal windows to a small number of static algebra operations, completely eliminating the unnecessary overhead of hundreds of thousands of repeated runs of moving average filtering and state machines in pigeon flock iteration; In the formula, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; For eye movement segments s The tag for the true interaction intent is 1 if there is a true interaction intent, and 0 otherwise. This represents the total number of eye-tracking segments in the training dataset.

[0038] In one optional implementation, a moving average filter for calligraphy and painting appreciation interaction is configured based on the optimal moving average time window of the optimal parameter vector, and an interest evaluator for calligraphy and painting appreciation interaction is configured based on the optimal gaze duration weight and the optimal pupil dilation weight, including: S20151: Convert the optimal moving average time window of the optimal parameter vector into the number of sampling points, using the following formula: In the formula, The number of sampling points; It is a rounding function; The optimal moving average time window; The sampling frequency of the eye-tracking device (e.g., 90Hz); In this embodiment, since the sampling frequency of the eye-tracking device is usually fixed (e.g., 90Hz, 120Hz), it is necessary to accurately convert physical time into the number of sampling points required for discrete digital signal processing, for example, optimally... =200ms, then =18 points, which will serve as the fixed length of the sliding window in step S203; S20152: Configure the number of sampling points to the size of the filter window of the moving average filter in the calligraphy and painting appreciation interaction; S20153: Based on the optimal gaze duration weight and the optimal pupil dilation weight, configure the interest evaluation tool for calligraphy and painting appreciation interaction. The formula is as follows: In the formula, The real-time interest score obtained by the interest evaluation function of the interest evaluator; The optimal fixation duration weight and optimal pupil dilation weight are the optimal parameter vectors. The cumulative duration of user interaction with the locked area of ​​the calligraphy and painting exhibits; This serves as a reference value for duration normalization. This represents the current relative rate of change in pupil size. The dead zone threshold for small fluctuations is set to a very small positive constant (e.g., 0.02, which is a 2% rate of change). The upper limit of the pupil extreme value is set as the maximum reasonable physiological dilation ratio that a normal person can achieve when viewing a painting (e.g., 0.30, or 30%), and is used as the denominator for normalization; To maximize the function; because in actual exhibition halls, ambient light has slight flicker (such as the flicker of exhibit spotlights), which can cause meaningless pupil tremors of 2%~3%, the function is set to... This noise can be directly blocked; and Set to the normal physiological limit (30%), when encountering rare cases where pupil data abnormally spikes due to strong light reflection, the system utilizes... The function is truncated and normalized to prevent outliers from inflating the interest score, which greatly enhances the robustness of the edge algorithm under complex lighting conditions. In the formula, This is the absolute value of the real-time pupil diameter at the current moment after outlier removal and filtering of the real-time pupil diameter sequence; The baseline absolute value of the pupil diameter is the arithmetic mean calculated after outlier removal from the real-time pupil diameter sequence.

[0039] In one optional implementation, the real-time raw gaze point sequence is mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and a moving average filter is used to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence, including: S2031: Map the coordinates of each real-time original gaze point in the real-time original gaze point sequence to the texture coordinate system of the digital model of the calligraphy and painting exhibit, to obtain a real-time mapped gaze point sequence with mapped coordinates set. In this embodiment, the eye-tracking device typically transmits data via USB 3.0 or a dedicated local area network streaming protocol. Real-time eye-tracking interaction data includes not only the real-time raw gaze point sequence (usually the screen pixel coordinates output after averaging the values ​​of one or both eyes), but also the real-time pupil diameter sequence. During the receiving phase, the underlying driver of the eye-tracking digital terminal needs to perform hardware-level timestamp alignment, that is, to unify the gaze point frame rate and pupil frame rate onto the same time axis using an interpolation algorithm (e.g., uniformly padding to 90Hz). If data frame loss due to user blinking is detected (pupil diameter suddenly changes to 0 or NaN), an "invalid frame" mark needs to be added at the underlying level and not included in subsequent cumulative duration calculations, ensuring data cleanliness from the data source. Since digital paintings and calligraphy may exist in the form of 3D scrolls, 3D frames or naked-eye 3D stereoscopic paintings, screen pixel coordinates cannot be directly used for appreciation and judgment. It is necessary to use the camera's intrinsic and extrinsic parameter matrix and the depth map of the image, and use the ray tracing method to back-project the 2D pixel points onto the ray in 3D space, find the intersection point with the surface of the digital painting and calligraphy model, and then extract the corresponding UV texture coordinates at the intersection point. S2032: Using a moving average filter, extract the filter window preceding the current time step from the real-time mapped gaze point sequence. Real-time mapped gaze point sequence segments; S2033: Calculate the smoothed coordinates of each real-time mapped gaze point in the real-time mapped gaze point sequence segment to obtain the user's real-time smoothed gaze point sequence. The formula is as follows: In the formula, For a moment k Smooth coordinates of each real-time mapped gaze point; In the filtering window Within the range, the first j The real-time mapping coordinates of the gaze point; k Index for the current time; j This is the fixation point indicator. For dynamic decay weight kernel; Traditional equal-weighted moving average filtering can cause a "tailing" effect in the gaze trajectory (i.e., the user has looked at a new location, but the calculated smoothing point still remains near the old location). A dynamically decaying weight kernel is used to address this issue. j = W When (the latest gaze point) is reached, the index is... =1, assigning the maximum weight, when j When =1 (oldest historical viewpoint), the index is Since 0 < <1, with W As the value increases, it approaches 0, giving it a very small weight. This creates a "top-heavy, bottom-light" weight distribution, making the eye-tracking coordinates closer to the current moment more decisive for the smoothing result. This asymmetric weighting mechanism ensures both the ability to filter out high-frequency noise and the "responsiveness" of the smoothing trajectory to the user's actual gaze shift, preventing visual misalignment when moving through details of calligraphy and painting (such as fine lines in meticulous brushwork).

[0040] In one optional implementation, an interest evaluator is used to assess interest based on a real-time smoothed gaze point sequence and a real-time pupil diameter sequence. Based on the obtained real-time interest score, the user's interaction lock-in area for the calligraphy and painting exhibits and its cumulative duration are determined, including: S2041: Based on the real-time smooth fixation point sequence, extract the real-time pupil diameter sequence segment from the real-time pupil diameter sequence; In this embodiment, since there is a physiological delay of tens to hundreds of milliseconds between pupil changes and eye movements, time window offset compensation is required when extracting real-time pupil diameter sequence segments (i.e., taking pupil data that is pushed forward several milliseconds from the current fixation point) to ensure that the fixation behavior and pupil dilation behavior have a strict causal correspondence in physical logic. S2042: Based on the ray method, determine whether the smoothed coordinates of each real-time smoothed gaze point in the real-time smoothed gaze point sequence fall within any hot zone. If so, count the cumulative duration of the gaze hot zone. In this embodiment, the backend server pre-marks polygonal hotspots on the digital model of calligraphy and painting. The terminal uses the ray method to determine whether the smoothed coordinates are inside the polygons. At the same time, a "tolerance mechanism" is introduced. Considering that the filtered coordinates may still have sub-pixel level errors, a buffer zone of 2-3 pixels is extended outward from the polygon boundary to avoid the situation where the user can see the boundary but cannot accumulate the time. S2043: Calculate the real-time relative pupil change rate based on the real-time pupil diameter sequence segment and the cumulative duration of the fixation hot zone, and calculate the real-time interest score of the fixation hot zone using the interest evaluator based on the real-time relative pupil change rate. S2044: If the real-time interest score is greater than the interest score threshold, the corresponding gaze hotspot will be used as the interaction lock area, and the user's interaction lock area for the calligraphy and painting exhibits and its cumulative duration will be output. In this embodiment, the baseline pupil diameter is not calculated as a fixed value, but rather by using a sliding baseline algorithm (such as taking the arithmetic mean of all valid pupil diameters within 5 seconds before the user enters the exhibit's field of vision). This eliminates the influence of differences in the natural pupil size of different users. The state machine flips from "gazing state" to "interactive lock state" only when the real-time interest score is greater than the interest score threshold, and outputs the ID of the hot zone and the value accurate to milliseconds. In one optional implementation, based on the cumulative duration, an interaction trigger command for the user's interaction-locked area is generated, sent to the backend server, and the backend server executes the calligraphy and painting appreciation assistance behavior corresponding to the interaction trigger command, including: S2051: Determine whether the cumulative duration of the interaction lock area is greater than the gaze threshold. If so, when the three-segment anti-shake state machine meets the conditions, use the eye-tracking digital terminal to encapsulate and generate an interaction trigger instruction with extremely low load. The interaction trigger instruction includes a timestamp, user ID, artwork ID, and area hash value. In this embodiment, if the terminal directly uploads the eye-tracking video stream or the original coordinate array, it will cause local area network congestion. The trigger command is compressed and refined into a trigger command that only occupies a few tens of bytes. The three-stage anti-shake state machine is the core to prevent accidental touch: State 1 (entering the hot zone and starting the timer) → if leaving the hot zone for a short time, enter State 2 (grace period, time not cleared) → if returning to the original hot zone within the specified time and the cumulative time is greater than the gaze threshold, enter State 3 (confirmed trigger). This perfectly solves the pain point of "interaction interruption" caused by users being attracted by people next to them and momentarily looking away when viewing calligraphy and paintings. S2052: Sends the interaction trigger command to the backend server through the preset private TCP / UDP port within the local area network; S2053: On the background server, the timeliness and debouncing and anti-replay checks of the interactive trigger command are performed. If the checks pass, proceed to the next step. In this embodiment, transmission is carried out through a private TCP / UDP port. After receiving the instruction, the backend extracts the timestamp and compares it with the current server clock. If the time difference exceeds the set threshold (e.g., 3 seconds), it is determined to be an expired packet and discarded. The hash value is extracted and compared in the cache queue. If it already exists, it is determined to be a replay attack and discarded, ensuring that the appreciation assistance behavior is not triggered repeatedly. S2054: Parse the interactive trigger command, use the local high-performance GPU mounted on the backend server to execute at least one calligraphy and painting appreciation assistance behavior, generate the corresponding data packet, and send the data packet to the eye-tracking digital terminal. The calligraphy and painting appreciation assistance behavior includes backend spatial audio mixing and distribution, backend dynamic image real-time synthesis and push, and backend collaborative knowledge graph fast retrieval. In this embodiment, the background spatial audio mixing is distributed: if the locked area is a "high mountains and flowing water" scene, the background GPU calls audio engines such as Wwise or FMOD to generate 3D ambient sound with Doppler effect and head-related transfer function (HRTF) in real time, so that users can feel as if they are there when they hear the sound. Background dynamic texture synthesis and push: If the locked area is a "damaged seal or faded ink", the background GPU uses an artificial intelligence repair model to generate a high-definition restoration texture in real time and pushes it through the video stream protocol; Backend collaborative knowledge graph fast retrieval: If the locked area is a specific person, the backend queries the Neo4j graph database to retrieve the person's social relationships, historical background and other network knowledge, and packages it into structured JSON data; S2055: Uses an eye-tracking digital terminal to call the local rendering engine to render and present data packets, and continuously monitors the user's latest eye-tracking interaction data, returning the gaze point filtering steps; In this embodiment, after the eye-tracking digital terminal receives the audio stream, texture stream, or JSON data, it calls the local rendering engine of Unity or UE to smoothly present the data in the form of AR overlay, picture-in-picture, or spatial audio without interrupting the rendering of the main screen. Most importantly, the system does not block during the presentation, but instead "continuously monitors the latest eye-tracking interaction data and returns to the gaze point filtering step (S203)," thereby forming a seamless immersive interaction loop.

[0041] This invention also provides an interactive device 300 for appreciating calligraphy and painting based on eye-tracking, as described in the embodiments of the present invention. Figure 3 The device may include the following units: The parameter deployment unit 301 is used to optimize parameters on the background server using an improved pigeon flock optimization algorithm, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal. The eye-tracking unit 302 is used to receive real-time eye-tracking interaction data of the user on the calligraphy and painting exhibits collected by the eye-tracking device in the eye-tracking digital terminal. The real-time eye-tracking interaction data includes a real-time original gaze point sequence and a real-time pupil diameter sequence. The gaze point filtering unit 303 is used to map the real-time raw gaze point sequence to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and to use a moving average filter to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence. The interaction locking unit 304 is used to evaluate interest based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence using an interest evaluator, and to determine the user's interaction locking area and its cumulative duration for the calligraphy and painting exhibits based on the obtained real-time interest score. The interaction triggering unit 305 is used to generate an interaction triggering command for the user's interaction lock area based on the cumulative duration, send it to the backend server, and execute the calligraphy and painting appreciation auxiliary behavior corresponding to the interaction triggering command through the backend server.

[0042] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; When the processor executes the program stored in the memory, it implements the eye-tracking-based interactive method for appreciating calligraphy and painting according to the present invention.

[0043] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EI) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM), or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0044] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0045] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the eye-tracking-based interactive method for appreciating calligraphy and painting according to embodiments of the present invention.

[0046] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0048] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0049] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0050] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0051] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An eye movement capture-based calligraphy and painting appreciation interaction method, characterized in that, The method includes: On the backend server, the improved pigeon flock optimization algorithm is used to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal. The eye-tracking digital terminal receives real-time eye-tracking interaction data of the user with respect to the calligraphy and painting exhibits, which is collected by the eye-tracking device. The real-time eye-tracking interaction data includes real-time raw fixation point sequence and real-time pupil diameter sequence. The real-time raw gaze point sequence is mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and the obtained real-time mapped gaze point sequence is filtered using a moving average filter to obtain the user's real-time smooth gaze point sequence. Based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence, an interest evaluator is used to assess interest, and based on the obtained real-time interest score, the user's interaction lock area for the calligraphy and painting exhibits and its cumulative duration are determined. Based on the cumulative duration, the system generates an interaction trigger command for the user's interaction-locked area, sends it to the backend server, and executes the corresponding calligraphy and painting appreciation assistance behavior through the backend server.

2. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 1, characterized in that, On the backend server, an improved pigeon flock optimization algorithm is used to optimize parameters, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal, including: On the backend server, historical eye-tracking interaction data of several users with calligraphy and painting exhibits is collected. The historical eye-tracking interaction data includes historical original fixation point sequences, historical pupil diameter sequences, and real interaction intent labels. Extract the static feature vector of each historical eye-tracking interaction data and add the static feature vector to the corresponding historical eye-tracking interaction data to construct a training dataset; The parameter vector consisting of the moving average time window, gaze duration weight, and pupil dilation weight is encoded into the position vector of an individual in the improved pigeon flock optimization algorithm. Construct a fitness function, and based on the training dataset, use the fitness function and an improved pigeon flock optimization algorithm to optimize the parameters and obtain the optimal parameter vector; Based on the optimal moving average time window of the optimal parameter vector, configure the moving average filter for calligraphy and painting appreciation interaction, and based on the optimal gaze duration weight and the optimal pupil dilation weight, configure the interest evaluation function for calligraphy and painting appreciation interaction. Deploy the moving average filter and interest evaluator to all eye-tracking digital terminals connected to the backend server.

3. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 2, characterized in that, Construct a fitness function, and based on the training dataset, use the fitness function and an improved pigeon flock optimization algorithm to optimize the parameters, obtaining the optimal parameter vector, including: Construct a fitness function, generate a chaotic sequence using a Logistic map, and map the chaotic sequence to the parameter space of individuals in the improved pigeon flock optimization algorithm to obtain the initial population. Based on the training dataset, the fitness function is used to calculate the fitness value of each initial individual in the initial population, and the initial individual with the best fitness value is taken as the optimal individual. The initial population is updated using the map compass operator to obtain an updated population. Based on the training dataset, the fitness function is used to calculate the fitness value of each updated individual in the population in one update, and the individual with the best fitness value in one update is updated as the best individual. Based on the fitness value, the bottom half of the individuals in the updated population are eliminated, and the center position of the remaining population is calculated. Based on the center position, the remaining population is updated to obtain a second-updated population; Based on the training dataset, the fitness function is used to calculate the fitness value of each individual in the population that is updated twice, and the individual with the best fitness value is updated as the best individual. Perform Cauchy mutation on the best individual to generate a trial individual, and add the trial individual to the population updated in the second iteration as the population to be updated in the next iteration; The position of the population is repeatedly updated. When the number of iterations reaches the maximum number of iterations or the fitness value of the best individual meets the requirements, the iterative optimization of the population is terminated, the position vector of the best individual is output, and the position vector of the best individual is decoded to obtain the optimal parameter vector.

4. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 3, characterized in that, The static feature vector includes the original average pupil change rate and the actual effective fixation frame number of any continuous eye movement segment falling within the local hot zone of the calligraphy and painting in the training dataset.

5. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 4, characterized in that, The formula for the fitness function is: In the formula, For individuals X The fitness values ​​of the corresponding alternative parameter vectors in the training dataset; For individuals X The prediction accuracy and false alarm rate of the corresponding alternative parameter vectors on the training dataset; This refers to the fitness weighting coefficient.

6. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 5, characterized in that, Based on the optimal moving average time window of the optimal parameter vector, configure the moving average filter for calligraphy and painting appreciation interaction, and based on the optimal gaze duration weight and optimal pupil dilation weight, configure the interest evaluation function for calligraphy and painting appreciation interaction, including: Convert the optimal moving average time window of the optimal parameter vector into the number of sampling points; Configure the number of sampling points to the size of the filter window of the moving average filter in the calligraphy and painting appreciation interaction. Configure an interest evaluator for calligraphy and painting appreciation interaction based on the optimal gaze duration weight and the optimal pupil dilation weight.

7. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 6, characterized in that, The real-time raw gaze point sequence is mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and a moving average filter is used to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence, including: The coordinates of each real-time original gaze point in the real-time original gaze point sequence are mapped to the texture coordinate system of the digital model of the calligraphy and painting exhibit, resulting in a real-time mapped gaze point sequence with mapped coordinates. Using a moving average filter, extract the filter window preceding the current time step from the real-time mapped gaze point sequence. Real-time mapped gaze point sequence segments; Calculate the smoothed coordinates of each real-time mapped gaze point in the real-time mapped gaze point sequence segment to obtain the user's real-time smoothed gaze point sequence.

8. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 7, characterized in that, Based on real-time smoothed fixation point sequences and real-time pupil diameter sequences, an interest evaluator is used to assess interest levels. Based on the obtained real-time interest scores, the user's interaction lock-on area for the calligraphy and painting exhibits and its cumulative duration are determined, including: Based on the real-time smoothed fixation point sequence, extract the real-time pupil diameter sequence segment from the real-time pupil diameter sequence; Based on the ray method, it is determined whether the smoothed coordinates of each real-time smoothed gaze point in the real-time smoothed gaze point sequence fall within any hot zone. If so, the cumulative duration of the gaze hot zone is counted. Based on the real-time pupil diameter sequence segment and the cumulative duration of the fixation hot zone, the real-time relative pupil change rate is calculated, and based on the real-time relative pupil change rate, the interest assessment tool is used to calculate the real-time interest score of the fixation hot zone. If the real-time interest score is greater than the interest score threshold, the corresponding gaze hotspot will be used as the interaction lock area, and the user's interaction lock area for the calligraphy and painting exhibits and its cumulative duration will be output.

9. The interactive method for appreciating calligraphy and painting based on eye-tracking capture according to claim 8, characterized in that, Based on the cumulative duration, an interaction trigger command for the user's interaction-locked area is generated and sent to the backend server. The backend server then executes the corresponding calligraphy and painting appreciation assistance behavior, including: Determine whether the cumulative duration of the interaction lock area is greater than the gaze threshold. If so, when the three-segment anti-shake state machine meets the conditions, use the eye-tracking digital terminal to encapsulate and generate an interaction trigger instruction with extremely low load. The interaction trigger instruction includes a timestamp, user ID, artwork ID, and area hash value. The interaction trigger command is sent to the backend server through a pre-set private TCP / UDP port within the local area network; On the backend server, the timeliness of the interactive trigger command is checked, as well as the debouncing and anti-replay checks. If the checks pass, proceed to the next step. The system analyzes interactive trigger commands, uses the local high-performance GPU mounted on the backend server to execute at least one calligraphy and painting appreciation assistance behavior, generates corresponding data packets, and sends the data packets to the eye-tracking digital terminal. The calligraphy and painting appreciation assistance behavior includes backend spatial audio mixing and distribution, backend dynamic image real-time synthesis and push, and backend collaborative knowledge graph rapid retrieval. Using an eye-tracking digital terminal, the local rendering engine is invoked to render and present data packets, and the latest eye-tracking interaction data of the user is continuously monitored to return the gaze point filtering steps.

10. A calligraphy and painting appreciation interactive device based on eye-tracking, used to implement the calligraphy and painting appreciation interactive method as described in any one of claims 1-9, characterized in that, The device includes: The parameter deployment unit is used on the backend server to optimize parameters using an improved pigeon flock optimization algorithm, complete the parameter configuration of the moving average filter and interest evaluator for calligraphy and painting appreciation interaction, and deploy them to the eye-tracking digital terminal. An eye-tracking unit is used to receive real-time eye-tracking interaction data of users with respect to calligraphy and painting exhibits collected by an eye-tracking device in an eye-tracking digital terminal. The real-time eye-tracking interaction data includes a real-time raw fixation point sequence and a real-time pupil diameter sequence. The gaze point filtering unit is used to map the real-time raw gaze point sequence to the texture coordinate system of the digital model of the calligraphy and painting exhibit, and to use a moving average filter to filter the obtained real-time mapped gaze point sequence to obtain the user's real-time smooth gaze point sequence. The interactive locking unit is used to evaluate interest based on the real-time smooth gaze point sequence and the real-time pupil diameter sequence using an interest evaluator, and to determine the user's interactive locking area and its cumulative duration for the calligraphy and painting exhibits based on the obtained real-time interest score. The interaction trigger unit is used to generate an interaction trigger command for the user's interaction lock area based on the cumulative duration, send it to the backend server, and execute the calligraphy and painting appreciation auxiliary behavior corresponding to the interaction trigger command through the backend server.

Images