Intelligent interaction experience optimization method and system combined with affective computing

Abstract

The application provides an intelligent interaction experience optimization method and system combined with emotional computing, and belongs to the technical field of intelligent interaction. First, original behavior data flow during multi-round task conversation of a user and a smart terminal is collected, including interface touch pressure distribution trajectory, voice instruction frequency spectrum feature sequence and eye gaze heat map change record. Then, the interface touch pressure distribution trajectory and the eye gaze heat map change record are double-mode fusion coded to generate an operation intention focus area distribution graph. The space emotion distribution atlas is obtained by superimposing the voice emotion color label, and the emotional negative experience blind area is identified therefrom. Finally, interface element strengthening prompt instructions are generated according to the blind area position, the blind area interface element is triggered to visually protrude, and the intelligent interaction experience is optimized.

Description

Technical Field

[0001] This application relates to the field of intelligent interaction technology, and more specifically, to a method and system for optimizing intelligent interactive experiences by combining affective computing. Background Technology

[0002] In the field of intelligent interaction, improving user experience has always been a core pursuit. Currently, most intelligent interaction systems mainly focus on function implementation and task completion, with relatively insufficient attention paid to the user's emotional experience. During multi-turn task dialogues between users and intelligent terminals, users' behaviors and emotional states are complex and ever-changing.

[0003] While some existing technologies attempt to collect user behavior data, such as focusing on single-modal data like touch pressure or voice commands, these individual data points struggle to comprehensively reflect a user's true intentions and emotional state. For example, relying solely on touch pressure cannot accurately determine whether changes in pressure are due to operational errors or disinterest in interface elements; analyzing only the semantics of voice commands fails to capture the underlying emotional inclinations in user speech. Furthermore, the lack of analytical methods that combine user operational intentions with emotional states makes it impossible to accurately pinpoint negative experience areas during interaction, hindering targeted optimization of the user interface and ultimately preventing substantial improvements in user experience. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and system for optimizing intelligent interactive experience by combining affective computing.

[0005] According to a first aspect of this application, a method for optimizing intelligent interactive experiences by incorporating affective computing is provided, the method comprising:

[0006] The raw behavioral data stream generated synchronously by the user during multi-round task dialogue with the smart terminal is collected. The raw behavioral data stream includes the interface touch pressure distribution trajectory, the voice command spectral feature sequence, and the eye gaze heatmap change record.

[0007] The interface touch pressure distribution trajectory and the eye gaze heatmap change record are subjected to dual-modal fusion encoding processing to generate a distribution map of the user's operation intention focus area on the interface;

[0008] Based on the spectral feature sequence of the voice command, a voice emotion color mark is superimposed on the distribution map of the operation intention focus area to obtain a spatial emotion distribution map that integrates operation intention and voice emotion.

[0009] The interface regions with negative voice emotion markers and operation intention density below a preset threshold are identified from the spatial emotion distribution map and marked as negative emotion experience blind spots.

[0010] Based on the location coordinates of the negative emotional experience blind spot in the spatial emotional distribution map, an interface element enhancement prompt instruction is generated for the negative emotional experience blind spot, and the interface element enhancement prompt instruction is sent to the interactive terminal to trigger the visual protrusion display operation of the interface element in the blind spot.

[0011] According to a second aspect of this application, an intelligent interactive experience optimization system incorporating affective computing is provided. The system includes a machine-readable storage medium and a processor. The machine-readable storage medium stores machine-executable instructions. When the processor executes the machine-executable instructions, the system implements the aforementioned intelligent interactive experience optimization method incorporating affective computing.

[0012] Based on any of the above aspects, the technical effect of this application is as follows:

[0013] By collecting raw behavioral data streams including interface touch pressure distribution trajectories, voice command spectral feature sequences, and eye gaze heatmap changes, dual-modal fusion encoding of the interface touch pressure distribution trajectory and eye gaze heatmap changes is performed to generate an operation intent focus area distribution map, accurately locating the concentrated areas of user operation intent on the interface. Based on this, a spatial emotion distribution map is obtained by overlaying voice emotion color markers, achieving an organic combination of operation intent and voice emotion, and intuitively presenting the correlation between the user's emotional state and operation intent during the interaction process. Negative emotional experience blind spots are identified from the spatial emotion distribution map, and interface element enhancement prompts are generated to trigger visual highlighting operations, specifically optimizing the interactive interface, effectively improving negative user experiences during the interaction process, and significantly enhancing the overall experience quality of intelligent interaction. Attached Figure Description

[0014] Figure 1 A flowchart illustrating the intelligent interactive experience optimization method combining affective computing provided in an embodiment of this application is shown.

[0015] Figure 2 This illustration shows a schematic diagram of the component structure of the intelligent interactive experience optimization system that combines emotion computing, as provided in an embodiment of this application. Detailed Implementation

[0016] Figure 1 This paper illustrates a flowchart of a method and system for optimizing intelligent interactive experiences by incorporating affective computing, as provided in an embodiment of this application. The detailed steps include:

[0017] Step S110: Collect the raw behavioral data stream generated synchronously by the user during multi-round task dialogue with the smart terminal. The raw behavioral data stream includes the interface touch pressure distribution trajectory, voice command spectrum feature sequence, and eye gaze heatmap change record.

[0018] The central control terminal synchronously collects three types of raw data through built-in sensors. Regarding the acquisition of interface touch pressure distribution trajectory, a pressure-sensitive touch sensor array captures the pressure distribution data generated by the user's finger contacting the screen at a preset sampling frequency. For each touch point, the sensor records the screen's horizontal and vertical coordinates, the pressure sensing value, and binds and encapsulates it with a timestamp generated by a high-precision clock, forming a raw touch point data unit. A series of units arranged in chronological order constitute the interface touch pressure distribution trajectory. Regarding the acquisition of voice command spectral feature sequences, a microphone array captures voice commands, performs analog-to-digital conversion, windowing and framing the digital audio stream. A Fast Fourier Transform is applied to each frame of audio signal to extract spectral features, representing them as a multi-dimensional vector. Each dimension corresponds to the energy amplitude at a preset frequency point. Vectors arranged in chronological order constitute the voice command spectral feature sequence. Regarding the acquisition of eye gaze heatmap change records, an infrared eye-tracking sensor captures the user's eye reflection image at a fixed frequency. The processing unit calculates the gaze point coordinates and gaze duration in real time. Each frame of data contains the gaze point coordinates and the corresponding duration, forming an eye gaze heatmap change record over time. All raw data is anonymized before entering the processing flow, a temporary identifier is generated for this interactive session, the three types of data are associated with the identifier, and the original image and audio waveform data are discarded.

[0019] Step S120: Perform dual-modal fusion encoding processing on the interface touch pressure distribution trajectory and the eye gaze heatmap change record to generate a distribution map of the user's operation intention focus area on the interface.

[0020] Step S121: Parse the interface touch pressure distribution trajectory from the original behavior data stream. The interface touch pressure distribution trajectory consists of multiple touch points. Each touch point includes the screen horizontal coordinate, screen vertical coordinate, pressure sensing value, and timestamp.

[0021] The touch modal data packets are filtered and extracted from the raw behavior data stream based on the data packet header type identifier. Each data packet is parsed to reconstruct each touch point that constitutes the touch pressure distribution trajectory of the interface. Each touch point is a four-tuple data structure containing four fields: screen horizontal coordinate, screen vertical coordinate, pressure sensing value, and timestamp. All touch points are organized into an ordered list according to the timestamp order, which is the parsed interface touch pressure distribution trajectory.

[0022] Step S122: Sort all touch points in the interface touch pressure distribution trajectory according to the timestamp order to generate an ordered touch point sequence. Select touch points with pressure sensing values ​​greater than the basic pressure threshold from the ordered touch point sequence as valid operation touch points. Count the number of valid operation touch points falling into each screen pixel grid to obtain the number of valid touch points in each screen pixel grid. Divide the number of valid touch points in each screen pixel grid by the sum of the number of valid touch points in all screen pixel grids to obtain the touch operation probability density value corresponding to each screen pixel grid. Based on the touch operation probability density value corresponding to each screen pixel grid, draw an initial touch density distribution map in the screen pixel grid coordinate system. The gray value of each pixel in the initial touch density distribution map is proportional to the touch operation probability density value of its grid.

[0023] Based on the timestamp field, all parsed touch points are globally sorted using a sorting algorithm, generating an ordered sequence of touch points arranged strictly in ascending order of time. A basic pressure threshold is set to distinguish between active and unintentional touches. The ordered touch point sequence is traversed, and the pressure sensitivity value of each touch point is compared with the basic pressure threshold. If it is greater, the touch point is considered a valid operation and is retained; otherwise, it is discarded. The screen display area is divided into multiple equal-sized screen pixel grids according to a preset resolution, each grid being uniquely identified by a row index and a column index. For each valid operation touch point, the row index and column index of its corresponding grid are calculated based on its horizontal and vertical screen coordinates. A two-dimensional counting matrix with the same dimensions as the screen pixel grid is maintained in memory. Whenever a valid operation touch point falls into a grid, the corresponding count value in the two-dimensional counting matrix is ​​incremented by one unit. After traversing all valid operation touch points, the two-dimensional counting matrix stores the number of valid touch points for each grid. The total number of valid touch points is obtained by summing all elements of the two-dimensional counting matrix. For each element of the two-dimensional counting matrix, i.e., the number of valid touch points in each grid, the number is divided by the total number of valid touch points to obtain the touch operation probability density value of that grid. The touch operation probability density values ​​of all grids form a two-dimensional probability density matrix with the same dimensions as the screen pixel grid. A blank grayscale image is constructed that completely corresponds to the screen pixel grid coordinate system, with a width equal to the number of grid columns and a height equal to the number of grid rows. For each element of the two-dimensional probability density matrix, for the grid with row index i and column index j, its touch operation probability density value is linearly mapped to a grayscale value. The grayscale value is equal to the touch operation probability density value multiplied by the maximum grayscale coefficient. The grayscale value is assigned to the pixel located at coordinates (i,j) in the blank grayscale image. The grayscale image obtained after traversing all grids is the initial touch density distribution map.

[0024] Step S123: Parse the eye gaze heatmap change record from the original behavior data stream. The eye gaze heatmap change record contains a continuous time frame sequence. Each time frame records the set of eye gaze point coordinates and the gaze duration corresponding to each eye gaze point.

[0025] Eye-tracking modal data packets are filtered and extracted from the raw behavioral data stream based on the packet header type identifier. Each eye-tracking data packet corresponds to a time frame. Each eye-tracking data packet is parsed to obtain all fixation point information within that time frame. The information of each time frame is organized into a structure containing the time frame timestamp and a fixation point list. Each fixation point in the fixation point list contains two fields: the screen horizontal and vertical coordinates of the fixation point and the fixation duration. All time frames are arranged in order of timestamp to obtain the eye fixation heatmap change record.

[0026] Step S124: Map the set of eye gaze point coordinates for each time frame to the screen pixel grid coordinate system. Based on the gaze duration length corresponding to each eye gaze point, accumulate the gaze duration for the screen pixel grid where the eye gaze point is located to generate the cumulative gaze duration value for each screen pixel grid. Normalize the cumulative gaze duration values ​​of all screen pixel grids to obtain the normalized gaze duration density value corresponding to each screen pixel grid.

[0027] The screen display area is divided into equal-sized screen pixel grids using the same screen pixel grid division method as in step S123. Each time frame in the eye gaze heatmap change record is traversed, and then each gaze point within each time frame is traversed. For each gaze point, the row and column indices of its corresponding screen pixel grid are calculated based on its horizontal and vertical screen coordinates. The gaze duration of that gaze point is accumulated into the cumulative time accumulator corresponding to that grid. After processing all time frames and all gaze points, the value stored in the cumulative time accumulator of each grid is the cumulative gaze time value for that grid. The maximum cumulative gaze time value among all grids is found. For each grid, its cumulative gaze time value is divided by this maximum value; the quotient is the normalized gaze time density value for that grid. The normalized gaze time density values ​​of all grids constitute a two-dimensional density matrix with the same dimensions as the screen pixel grid.

[0028] Step S125: Based on the normalized gaze time density value, draw an initial gaze density distribution map in the screen pixel grid coordinate system. The gray value of each pixel in the initial gaze density distribution map is proportional to the normalized gaze time density value of the grid it belongs to.

[0029] Construct another blank grayscale image that corresponds perfectly to the screen pixel grid coordinate system, with a width equal to the number of grid columns and a height equal to the number of grid rows. Iterate through each element of the two-dimensional density matrix obtained in step S124. For the grid with row index p and column index q, linearly map its normalized gaze time density value to a grayscale value. The grayscale value is equal to the normalized gaze time density value multiplied by the maximum grayscale coefficient. Assign the grayscale value to the pixel located at coordinates (p, q) in the blank grayscale image. The grayscale image obtained after traversing all grids is the initial gaze density distribution map.

[0030] Step S126: Perform spatial registration processing on the initial touch density distribution map and the initial gaze density distribution map so that the initial touch density distribution map and the initial gaze density distribution map use the same screen pixel grid coordinate system and the same grid resolution.

[0031] Since the initial touch density distribution map generated in step S125 and the initial gaze density distribution map generated in step S128 are both drawn based on the screen pixel grid coordinate system divided by the same screen display area, and the grid resolution is set to the same value at the beginning of the division, the two naturally have the same spatial reference system and resolution, and spatial registration can be completed without additional geometric transformation or resampling.

[0032] Step S127: The touch operation probability density value of each screen pixel grid in the initial touch density distribution map is weighted and summed with the normalized gaze time density value of the corresponding screen pixel grid in the initial gaze density distribution map to obtain the fusion intent density value of each screen pixel grid. Based on the fusion intent density values ​​of all screen pixel grids, a grayscale distribution map is redrawn in the screen pixel grid coordinate system. The grayscale distribution map is used as the distribution map of the user's operation intent focus area on the interface.

[0033] Set touch weight coefficients and gaze weight coefficients, with their sum equal to 1. Iterate through each screen pixel grid. For the grid with row index r and column index s, read its touch operation probability density value from the initial touch density distribution map and its normalized gaze time density value from the initial gaze density distribution map. Multiply the touch operation probability density value by the touch weight coefficient to obtain the touch weighted value, and multiply the normalized gaze time density value by the gaze weight coefficient to obtain the gaze weighted value. Add the touch weighted value and the gaze weighted value to obtain the fused intent density value for that grid. Perform the above operation on all grids to obtain a fused intent density matrix with the same dimensions as the screen pixel grid. Construct a third blank grayscale image that corresponds perfectly to the screen pixel grid coordinate system. The width is equal to the number of grid columns and the height is equal to the number of grid rows. Traverse each element of the fusion intent density matrix. For the grid with row index u and column index v, linearly map its fusion intent density value to a grayscale value. The grayscale value is equal to the fusion intent density value multiplied by the maximum grayscale coefficient. Assign the grayscale value to the pixel located at coordinates (u, v) in the blank grayscale image. The grayscale image obtained after traversing all grids is the distribution map of the operation intent focus area.

[0034] Step S130: Based on the spectral feature sequence of the voice command, overlay voice emotion color markers onto the distribution map of the operation intention focus area to obtain a spatial emotion distribution map that integrates operation intention and voice emotion.

[0035] Step S131: Parse the speech command spectral feature sequence from the original behavior data stream. The speech command spectral feature sequence consists of multiple speech frame spectral vectors arranged in chronological order. Each speech frame spectral vector contains the energy amplitude of the speech frame at multiple frequency points.

[0036] Voice modal data packets are filtered and extracted from the raw behavior data stream based on the data packet header type identifier. Each voice data packet corresponds to a voice frame. Each voice data packet is parsed to obtain the spectrum vector of the voice frame. The number of dimensions is the same as the number of preset frequency points. Each component in the vector represents the energy amplitude at the corresponding frequency point. All voice frame spectrum vectors are arranged in chronological order to obtain the voice command spectrum feature sequence.

[0037] Step S132: Perform speech emotion feature extraction processing on the speech frame spectrum vector of each speech frame in the speech command spectrum feature sequence, calculate the spectral centroid offset, spectral roll-off rate of change and spectral passband energy distribution ratio of each speech frame spectrum vector, and combine them to generate a three-dimensional speech emotion feature vector corresponding to each speech frame. Then, input it into a pre-trained speech emotion classifier to obtain three posterior probability values ​​output by the speech emotion classifier indicating that the speech frame belongs to the positive emotion category, neutral emotion category and negative emotion category respectively. Select the emotion category corresponding to the largest posterior probability value from the three posterior probability values ​​as the speech emotion category label of the speech frame.

[0038] The process iterates through the spectral vector of each speech frame in the spectral feature sequence of the speech command. For the current speech frame's spectral vector, first calculate the spectral centroid, which is equal to the sum of the energy amplitudes of all frequency points multiplied by their respective frequencies, divided by the sum of all energy amplitudes. Subtract the spectral centroid of the previous speech frame from the current spectral centroid to obtain the difference, which is the spectral centroid offset. Calculate the spectral roll-off point, defined as the frequency point where the sum of the energy amplitudes of all frequency points below that point represents a predetermined percentage of the total energy of the entire spectrum. Subtract the spectral roll-off point of the previous speech frame from the current spectral roll-off point to obtain the difference, which is then divided by the previous spectral roll-off point to obtain the spectral roll-off point change rate. Divide the entire spectrum into multiple predetermined frequency bands, calculate the sum of the energy amplitudes of all frequency points within each band, and then divide the sum of the energy in each band by the total energy of the entire spectrum. The resulting ratios represent the spectral passband energy distribution ratio. The calculated spectral centroid offset, spectral roll-off rate of change, and the proportion of spectral passband energy distribution composed of multiple values ​​are used as the three components of a three-dimensional vector, and each speech frame is represented as a three-dimensional speech emotion feature vector. The pre-trained speech emotion classifier uses a deep neural network model, with the number of nodes in the input layer matching the dimension of the three-dimensional speech emotion feature vector. Multiple fully connected hidden layers are connected after the input layer, each containing multiple neurons and employing a non-linear activation function. The last hidden layer is connected to the output layer, which contains three neurons corresponding to positive, neutral, and negative emotion categories, respectively. A flexible maximum transfer function is used to make the sum of the three output values ​​equal to 1, representing the posterior probability of the input feature vector belonging to the corresponding emotion category. Each three-dimensional speech emotion feature vector is sequentially input into the pre-trained speech emotion classifier. The forward propagation passes through each hidden layer, and three posterior probability values ​​are obtained from the output layer. These three values ​​are compared, and the largest posterior probability value is found. Based on the correspondence between the neurons in the output layer, the emotion category represented by this largest posterior probability value is determined as the speech emotion category label for that speech frame.

[0039] Step S133: Based on the time order of each speech frame in the speech command spectrum feature sequence, assign a speech frame start timestamp and a speech frame end timestamp to each speech frame to form a speech emotion category label sequence with time interval information. Align the speech emotion category label sequence with time interval information with the operation intention focus area distribution map on the time axis. The operation intention focus area distribution map has a time axis synchronized with the original behavior data stream.

[0040] Each frame in the speech command spectral feature sequence corresponds to a time period in the original audio stream. Each speech frame is associated with a start time and an end time. The speech emotion category label of each speech frame obtained in step S132 is bound to its corresponding speech frame start timestamp and end timestamp to obtain a series of data units. Each unit contains three fields: speech frame start timestamp, end timestamp, and speech emotion category label. The above units are arranged in chronological order to form a speech emotion category label sequence with time interval information. The time axis of the above label sequence is compared with the global time axis used when collecting the original behavior data stream. Since all data collection uses a unified time source, the two are naturally synchronized.

[0041] Step S134: Traverse each voice emotion category label in the voice emotion category label sequence with time interval information, and extract the fusion intent density value distribution map corresponding to all time points in the time interval from the operation intent focus area distribution map according to the voice frame start timestamp and voice frame end timestamp of each voice emotion category label, and generate a series of time domain sub-distribution maps corresponding to the time interval.

[0042] For each label in the speech emotion category label sequence with time interval information, the start and end timestamps of its corresponding speech frame define a time period. Returning to step S127, the process of generating the fusion intent density matrix is ​​repeated, re-executing the parts involving the selection and statistics of effective operation touch points and gaze points from steps S122 to S127. However, only effective operation touch points and gaze points whose timestamps fall within the interval of the start and end timestamps of the current speech frame are used. A fusion intent density matrix based solely on data within that time period is generated for each speech emotion category label's time interval, and corresponding grayscale distribution maps are drawn based on this matrix. These grayscale maps are the temporal sub-distribution maps corresponding to a series of time intervals.

[0043] Step S135: Assign a speech emotion category label to each time domain sub-distribution map, wherein the speech emotion category label is the same as the speech emotion category label in the corresponding time interval of the time domain sub-distribution map.

[0044] For each time-domain sub-distribution map generated in step S134, based on the time interval used when generating the map, find the label corresponding to the time interval in the speech emotion category label sequence with time interval information, and assign the speech emotion category label in the label to the time-domain sub-distribution map as its speech emotion category label.

[0045] Step S136: The voice emotion category markers are superimposed on the corresponding temporal sub-distribution map in the form of a semi-transparent color layer to form a temporal sub-map with emotion color coverage. The temporal sub-maps with emotion color coverage are arranged in chronological order according to their corresponding time intervals, and adjacent temporal sub-maps are smoothly transitioned and merged at the time boundaries to eliminate the color abrupt boundary caused by the change in emotion category between temporal sub-maps, and generate a continuous emotion color gradient layer.

[0046] A pre-defined emotion color mapping table defines colors corresponding to positive, neutral, and negative emotion categories. For each temporal sub-distribution map, the corresponding color is looked up from the mapping table based on its speech emotion category label. A semi-transparent color layer with the same size as the temporal sub-distribution map is created, and this layer is completely filled with the found color. This layer is then overlaid on the original temporal sub-distribution map to form a temporal sub-map with emotion color overlay. All temporal sub-maps are arranged sequentially from left to right according to the start timestamp of their respective time intervals, forming a continuous image sequence on the timeline. At the seam between two adjacent temporal sub-maps, if the emotion categories are different, the color changes abruptly. To eliminate the abrupt change, a smooth transition fusion process is performed on the adjacent maps at the temporal boundary. A small segment of the image at the end of the previous map and a small segment of the image at the beginning of the next map are taken, and the color values ​​of the corresponding pixels in these two small segments are weighted and averaged. The weight changes linearly with the distance from the boundary. The closer to the boundary, the greater the weight of the next image. The transition image generated after weighted averaging replaces the original boundary area. After processing all adjacent boundaries, the discrete temporal sub-maps are merged into an emotion color gradient layer with continuously changing colors on the time axis.

[0047] Step S137: The emotional color gradient layer is overlaid and fused with the complete operation intention focus area distribution map, so that the color information in the emotional color gradient layer is accurately covered on the corresponding time area of ​​the operation intention focus area distribution map according to the time axis, resulting in a two-dimensional spatiotemporal image that contains both operation intention density information and voice emotional color information. The two-dimensional spatiotemporal image is post-processed, a time scale bar is added to the bottom of the two-dimensional spatiotemporal image, and a legend corresponding to the emotional color and emotional category is added to the right side of the two-dimensional spatiotemporal image, finally generating a spatial emotional distribution map that integrates operation intention and voice emotion.

[0048] An emotional color gradient layer is overlaid as an additional color channel on the complete operation intent focus area distribution map. This emotional color gradient layer is then tiled along the timeline below or above the operation intent focus area distribution map to form a two-dimensional image. The horizontal axis of the image represents both the screen's horizontal position and implicitly time, while the vertical axis represents the screen's vertical position. The grayscale value of each pixel comes from the operation intent focus area distribution map, representing the intent intensity at that screen position throughout the entire interaction period. The color value of each pixel comes from the emotional color gradient layer, representing the associated voice emotion at that specific moment during the interaction. A time scale is drawn below the bottom edge of the image, with the starting point corresponding to the start time of the interaction session and the ending point corresponding to the end time. Key time points are marked at equal intervals according to the timeline length. An emotional color and emotion category legend is drawn outside the right edge of the image, with vertical bars displaying the colors corresponding to positive, neutral, and negative emotion categories from top to bottom, along with their names. Finally, a spatial emotion distribution map integrating operation intent and voice emotion is generated.

[0049] Step S140: Identify interface regions with negative voice emotion markers and operation intention density below a preset threshold from the spatial emotion distribution map, and mark the interface regions as negative emotion experience blind spots.

[0050] Step S141: Analyze the spatial emotion distribution map, which is composed of a pixel matrix. Each pixel stores a grayscale value and a color channel value. The grayscale value represents the operation intent density at the location of the pixel, and the color channel value represents the voice emotion color mark superimposed at the location of the pixel.

[0051] The spatial emotion distribution map generated in step S137 is loaded into memory and represented as a pixel matrix. Each element of the matrix corresponds to a pixel in the map. Each pixel contains two sets of data: grayscale value and color channel value. The grayscale value represents the density of the operation intent at the screen position of the pixel during the entire interaction period. The color channel value is composed of the values ​​of the red, green and blue channels, representing the voice emotion color superimposed at the screen position.

[0052] Step S142: Traverse each pixel in the spatial emotion distribution map, read the color channel value of each pixel, and convert the color channel value of each pixel into the corresponding voice emotion category identifier according to the preset emotion color and emotion category lookup table.

[0053] The pixel matrix is ​​traversed using a nested loop, proceeding row-first and column-second. For each pixel, its color channel value (red, green, and blue components) is read from its data structure. A pre-defined table mapping emotional color to emotional category is used to divide the red-green-blue color space into several subspaces. If a pixel's red-green-blue channel value falls into the subspace representing negative emotion, it is classified as negative; if it falls into the subspace representing positive emotion, it is classified as positive; and if it falls into the subspace representing neutral emotion, it is classified as neutral. For each pixel's color channel value, its subspace is determined, and this is converted into a discrete voice emotion category identifier.

[0054] Step S143: Determine whether the voice emotion category identifier corresponding to each pixel is a negative emotion category identifier. If the voice emotion category identifier is a negative emotion category identifier, then mark the pixel as a negative emotion candidate pixel. For each pixel marked as a negative emotion candidate pixel, read its stored gray value and compare the gray value with a preset density threshold to determine whether the gray value is less than the preset density threshold.

[0055] For each pixel, the voice emotion category identifier is determined. If the identifier equals the value representing a negative emotion category, the pixel is marked as a negative emotion candidate pixel. A labeling matrix of the same size as the original atlas is maintained in memory, and a specific value is stored at the corresponding position in the labeling matrix. For each negative emotion candidate pixel in the labeling matrix, the grayscale value stored for that pixel is read from the pixel matrix of the spatial emotion distribution map. A preset density threshold is obtained to define the density of the operation intent. The grayscale value is then compared with the preset density threshold to determine whether the former is less than the latter.

[0056] Step S144: If the gray value of the negative emotion candidate pixel is less than the preset density threshold, the negative emotion candidate pixel is marked as a blind zone seed pixel, and the pixel row index and pixel column index of the blind zone seed pixel in the spatial emotion distribution map are recorded. Connectivity analysis is performed on all pixels marked as blind zone seed pixels, and adjacent blind zone seed pixels are aggregated into connected pixel sets. Each connected pixel set constitutes a candidate blind zone connected region.

[0057] For negative sentiment candidate pixels with grayscale values ​​less than a preset density threshold (i.e., pixels that are both negative sentiment candidates and have grayscale values ​​below the preset density threshold), they are identified as blind zone seed pixels. A specific value is stored at the corresponding position in a new labeling matrix, and the row and column indices of the pixel in the pixel matrix are recorded. A connected component labeling algorithm is used to process all blind zone seed pixels, checking each blind zone seed pixel and its eight neighboring pixels in the top, bottom, left, right, and diagonal directions. If neighboring pixels are also blind zone seed pixels, they are grouped into the same set. After traversing all blind zone seed pixels, several sets are output. Each set contains a group of spatially adjacent blind zone seed pixels, and each set constitutes a candidate blind zone connected region.

[0058] Step S145: Calculate the total number of pixels contained in each candidate blind zone connected region, divide the total number of pixels by the preset minimum blind zone area threshold, determine whether the candidate blind zone connected region meets the minimum area requirement, filter out candidate blind zone connected regions that do not meet the minimum area requirement, and for candidate blind zone connected regions that meet the minimum area requirement, extract the minimum pixel row index, maximum pixel row index, minimum pixel column index, and maximum pixel column index of all pixels in the candidate blind zone connected region.

[0059] For each candidate blind region, the number of seed pixels it contains is counted, which represents the area of ​​that region. A preset minimum blind region area threshold is obtained to filter out noise regions with excessively small areas. The total number of pixels is compared with the minimum blind region area threshold; if it is less than the threshold, it is considered a noise region and is filtered out and not included in subsequent processing; if it is greater than or equal to the threshold, it is retained. For each retained candidate blind region, all its pixels are traversed, and the minimum, maximum, minimum, and maximum values ​​of the row, column, and column indices of these pixels are recorded. These four values ​​represent the minimum and maximum boundaries in the vertical and horizontal directions of the connected region, respectively.

[0060] Step S146: Based on the minimum pixel row index, the maximum pixel row index, the minimum pixel column index, and the maximum pixel column index, draw an outer rectangle on the spatial sentiment distribution map. The four sides of the outer rectangle correspond to the row containing the minimum pixel row index, the row containing the maximum pixel row index, the column containing the minimum pixel column index, and the column containing the maximum pixel column index, respectively. The rectangular area enclosed by the outer rectangle in the spatial sentiment distribution map is determined as the preliminary boundary contour of the negative sentiment experience blind zone. The starting row coordinates, ending row coordinates, starting column coordinates, and ending column coordinates of the negative sentiment experience blind zone are recorded based on the preliminary boundary contour.

[0061] A rectangle is drawn on the spatial sentiment distribution map image based on four boundary values: the top edge is located in the row with the minimum pixel index, the bottom edge is located in the row with the maximum pixel index, the left edge is located in the column with the minimum pixel index, and the right edge is located in the column with the maximum pixel index. This rectangle is the minimum axis-aligned bounding rectangle enclosing the connected region of the candidate blind zone. The internal region enclosed by the bounding rectangle is defined as the preliminary boundary contour of the negative sentiment experience blind zone represented by the connected region of the candidate blind zone. The four boundary values ​​constituting the contour—the starting row coordinates, the ending row coordinates, the starting column coordinates, and the ending column coordinates—are recorded as preliminary positional information.

[0062] Step S147: Perform contour smoothing on the preliminary boundary contour, replace the four right angles of the outer rectangle with rounded corners with a certain radius of curvature to generate the final boundary contour of the emotional negative experience blind zone, and uniformly mark all pixels in the final boundary contour as emotional negative experience blind zone pixels to complete the marking of the emotional negative experience blind zone.

[0063] Morphological operations in image processing are used to smooth the initial boundary contour. A quarter-circle arc is drawn with each corner point as the center and a preset radius of curvature as the radius, replacing the original right angles and transforming the rectangular boundary into a rounded rectangle, which serves as the final boundary contour for the emotional negative experience blind spot. All pixels falling within the final boundary contour are found in the marker matrix, and their marker values ​​are uniformly modified to specific values ​​to indicate that they belong to the finally confirmed emotional negative experience blind spot.

[0064] Step S150: Based on the position coordinates of the negative emotional experience blind spot in the spatial emotional distribution map, generate an interface element enhancement prompt instruction for the negative emotional experience blind spot, and send the interface element enhancement prompt instruction to the interactive terminal to trigger the visual protrusion display operation of the interface element in the blind spot.

[0065] Step S151: Read the final boundary contour corresponding to each negative emotional experience blind spot from the spatial emotion distribution map. The final boundary contour is described by the starting row coordinates, ending row coordinates, starting column coordinates, and ending column coordinates. Convert the starting row coordinates and ending row coordinates into the starting vertical pixel position and ending vertical pixel position in the screen pixel coordinate system, and convert the starting column coordinates and ending column coordinates into the starting horizontal pixel position and ending horizontal pixel position in the screen pixel coordinate system to obtain the screen coordinate blind spot rectangle.

[0066] The final boundary contour information of each negative emotional experience blind spot is obtained from the processing results of steps S146 and S147. The final boundary contour of each blind spot is described by four parameters: the starting row coordinates, the ending row coordinates, the starting column coordinates, and the ending column coordinates. Since all previous processing is based on the screen pixel grid coordinate system, these coordinates are consistent with the actual physical screen pixel coordinate system of the terminal. The starting row coordinates and the ending row coordinates directly correspond to the starting vertical pixel position and the ending vertical pixel position in the screen pixel coordinate system, and the starting column coordinates and the ending column coordinates directly correspond to the starting horizontal pixel position and the ending horizontal pixel position. The rectangle determined by these four actual screen pixel positions is the screen coordinate blind spot rectangle.

[0067] Step S152: Calculate the center pixel coordinates of the screen coordinate blind zone rectangle based on the starting vertical pixel position, ending vertical pixel position, starting horizontal pixel position, and ending horizontal pixel position. The vertical coordinate of the center pixel coordinates is equal to half the sum of the starting vertical pixel position and the ending vertical pixel position, and the horizontal coordinate is equal to half the sum of the starting horizontal pixel position and the ending horizontal pixel position.

[0068] Calculate the geometric center of the screen coordinate blind zone rectangle. The vertical coordinate of the center point is equal to the sum of the starting and ending vertical pixel positions divided by 2. The horizontal coordinate of the center point is equal to the sum of the starting and ending horizontal pixel positions divided by 2. The two calculated values ​​constitute the pixel coordinates of the center point of the negative emotional experience blind zone.

[0069] Step S153: Obtain the interface layout description file of the current interface of the smart interactive terminal. The interface layout description file contains the element identifiers of all interface elements on the current interface and the element bounding box coordinates of each interface element on the screen. Calculate the overlapping area between the screen coordinate blind zone rectangle and the element bounding box coordinates of each interface element in the interface layout description file. Find all interface elements that have overlapping areas with the screen coordinate blind zone rectangle to form a blind zone associated interface element set. For each blind zone associated interface element in the blind zone associated interface element set, calculate the overlapping area between the element bounding box of the blind zone associated interface element and the screen coordinate blind zone rectangle.

[0070] The interface layout description file is retrieved from the current active window of the central control terminal. This file is a structured data file, which is parsed to obtain information on all interface elements on the current screen. Each interface element records a unique element identifier and its bounding box coordinates on the screen. Each interface element is traversed, and the intersection of its bounding box coordinates with the screen coordinate blind zone rectangle is calculated. If the area of ​​the intersection region is greater than 0, it is determined that the interface element has a spatial overlap relationship with the negative emotional experience blind zone. The element identifier and related information of this interface element are added to a temporary set to form a blind zone associated interface element set. For each element in the set, its overlap area with the screen coordinate blind zone rectangle is further precisely calculated. The overlap area is calculated based on the width and height of the intersection of the two rectangles. The width of the intersection rectangle is equal to the smaller right boundary value minus the larger left boundary value, and the height is equal to the smaller bottom boundary value minus the larger top boundary value. The overlap area is obtained by multiplying the width and height.

[0071] Step S154: Divide the overlapping area by the total area of ​​the bounding boxes of the interface elements associated with the blind zone to obtain the overlapping area ratio of the interface elements associated with the blind zone. Identify the interface elements associated with the blind zone whose overlapping area ratio exceeds a preset overlapping ratio threshold as interface elements mainly affected by the blind zone, and record the element identifier of the interface elements mainly affected by the blind zone.

[0072] The total area of ​​the bounding boxes of the associated interface elements in the blind spot is calculated as width multiplied by height. The overlapping area is then divided by the total area of ​​the element itself, resulting in a value between 0 and 1, which is the overlap area ratio. This quantifies the degree to which the interface element is covered by the emotionally negative experience blind spot. A preset overlap ratio threshold is set to determine whether an interface element is mainly located within the blind spot. For each element in the set of interface elements associated with the blind spot, its overlap area ratio is compared with the preset overlap ratio threshold. If the ratio is greater than the threshold, the main part of the interface element is considered to be located within the experience blind spot, making it the element most directly affected by the negative impact. The element identifiers of these elements are recorded to form a list of interface elements mainly affected by the blind spot.

[0073] Step S155: Based on the element identifier of the interface element mainly affected by the blind spot, retrieve the visual convexity display parameter set corresponding to the corresponding interface element from the preset function enhancement effect library. The visual convexity display parameter set includes edge highlight color value, edge highlight width value, and element scaling ratio value. Based on the element identifier of the interface element mainly affected by the blind spot, retrieve the function guidance prompt text content and function guidance prompt icon corresponding to the corresponding interface element from the preset guidance prompt content library. Encapsulate the visual convexity display parameter set, the function guidance prompt text content, and the function guidance prompt icon to generate an element-level enhancement prompt sub-instruction for a single blind spot associated interface element.

[0074] The preset enhancement effect library is a data structure that predefines multiple visual enhancement schemes based on interface element type or function. For each element identifier in the list of interface elements mainly affected by blind spots, the library is queried using this identifier as the primary key to obtain a set of visual protrusion display parameters associated with that identifier. This set includes an edge highlight color value (composed of red, green, and blue components defining the highlight border color), an edge highlight width value (defining the highlight border thickness), and an element scaling ratio value (defining the element's magnification factor). The preset guidance prompt content library stores functional description text and prompt icons for different interface elements. For each element identifier, the library is queried to obtain the functional guidance prompt text content and functional guidance prompt icon. The visual protrusion display parameter set, functional guidance prompt text content, and functional guidance prompt icon are assembled according to a predefined instruction format. The data structure includes an instruction type identifier, a target element identifier, and specific execution parameters to generate an element-level enhancement prompt sub-instruction for a single interface element.

[0075] Step S156: Repeat the above process to generate corresponding element-level enhancement prompt sub-instructions for each interface element mainly affected by the blind spot, and combine all element-level enhancement prompt sub-instructions into a complete interface element enhancement prompt instruction. Add the instruction version number and timestamp information to the header of the interface element enhancement prompt instruction, and add a check code to the tail of the interface element enhancement prompt instruction.

[0076] For each element identifier in the list of interface elements primarily affected by the blind spot, step S155 is repeated until a corresponding element-level enhanced hint sub-instruction is generated for all elements in the list. These element-level enhanced hint sub-instructions are then combined into a list in a predetermined order and encapsulated into a larger data structure to form a complete interface element enhanced hint instruction. A field storing the instruction version number (indicating the instruction format version) is added to the header of the data structure, and a field storing the current timestamp (recording the instruction generation time) is added. At the end, a checksum is calculated using a cyclic redundancy check algorithm based on the entire instruction content and added. After encapsulation, the final generated interface element enhanced hint instruction is sent to the central control terminal's interface rendering engine via inter-process communication, triggering visual highlighting operations such as edge highlighting, slight magnification, and the appearance of function prompt bubbles for interface elements within the blind spot.

[0077] Step S210: After generating the operation intent focus area distribution map, perform region segmentation processing on the operation intent focus area distribution map, identify regions in the operation intent focus area distribution map whose fusion intent density value is lower than a preset sparse density threshold, and designate them as density sparse regions. Assign different region type identifiers to each density sparse region, and attach the region type identifiers to the corresponding region pixels in the operation intent focus area distribution map.

[0078] After generating the distribution map of the focus area for the operation intent, the map is segmented. A preset sparse density threshold is set, and a global threshold segmentation method is used to traverse the grayscale value of each pixel in the distribution map. Pixels with grayscale values ​​below the threshold are marked as candidate pixels. Connectivity analysis is performed on the candidate pixels, aggregating adjacent candidate pixels into connected pixel sets, each set constituting a sparse region. A unique region type identifier is assigned to each sparse region. An additional field is added to the pixel matrix storing the distribution map for each pixel. The value of this field for all pixels within the sparse region is set to the corresponding region type identifier, while the field for pixels in non-sparse regions is set to null.

[0079] Step S220: When drawing voice emotion color markers by superimposing the spectral feature sequence of the voice command, the region type identifier covered by each voice emotion color marker is recorded simultaneously, the total area of ​​each voice emotion color marker covered by the region type identifier being a sparse region is calculated, and the total area is compared with a preset threshold for the area of ​​sparse emotional association regions.

[0080] When rendering the overlay of voice emotion color markers, a marker matrix of the same size as each temporal sub-map is created simultaneously. For each pixel in the temporal sub-map, its position coordinates are used to backtrack to the updated operation intent focus area distribution map. The region type identifier corresponding to that pixel is read, and if it is located in a sparse region, it is recorded at the corresponding position in the marker matrix. For each voice emotion color marker, its marker matrix is ​​traversed, and the number of pixels with non-empty region type identifiers is accumulated to obtain the total area of ​​the sparse region covered by that marker. This total area of ​​the sparse region is then numerically compared with a preset threshold for the area of ​​the emotion-related sparse region.

[0081] Step S230: If the total area exceeds the threshold of the area of ​​the sparse region of emotional association, the part of the sparse region that overlaps with the negative speech emotion color mark is preferentially marked as a candidate negative emotional experience blind zone. Temporal stability analysis is performed on the candidate negative emotional experience blind zone, and the frequency of occurrence of the candidate negative emotional experience blind zone in the spatial emotion distribution map of multiple consecutive time frames is extracted.

[0082] For the temporal sub-map corresponding to a speech emotion color marker whose total area exceeds the threshold of the sparse region of emotion association, if its speech emotion category is negative, then the pixels covered by the temporal sub-map with a non-empty region type identifier are marked as candidate negative emotion experience blind spots in the candidate label matrix. The location information of the candidate blind spots is recorded, and the process is traced back to multiple consecutive time frames used in generating the spatial emotion distribution map. For each subsequent time frame, the temporal sub-map is checked to see if it simultaneously satisfies the conditions of being located within a sparse region and carrying a negative emotion color marker. The number of times this condition is met is counted as the frequency of occurrence.

[0083] Step S240: If the frequency of occurrence of the candidate negative emotional experience blind zone in multiple consecutive time frames exceeds a preset frequency threshold, then the candidate negative emotional experience blind zone is confirmed as a negative emotional experience blind zone. The region information of the confirmed negative emotional experience blind zone is fed back to the generation process of the spatial emotion distribution map to adjust the transparency parameter of the voice emotion color marker in the subsequent spatial emotion distribution map.

[0084] A preset frequency threshold is set, and the statistically obtained occurrence frequency is compared with this threshold. If the occurrence frequency is greater than the threshold, it indicates that the region continuously exhibits operational sparseness and negative emotion superposition, and the candidate blind spot is officially confirmed as a negative emotional experience blind spot. The information of the confirmed blind spot is sent back to the spatial emotion distribution map generation module as a feedback signal. In subsequent processing, for pixels falling within the confirmed blind spot range, the transparency parameter of their color layer is adjusted to reduce transparency, making these areas more prominent in the final map.

[0085] Step S250: Based on the adjustment result of the transparency parameter, dynamically correct the boundary contour accuracy of the subsequently identified negative emotional experience blind zone, compare the corrected boundary contour of the negative emotional experience blind zone with the historical blind zone boundary contour stored in the history record, and if the similarity exceeds the preset similarity threshold, trigger the parameter update operation of the blind zone prediction model.

[0086] Based on the transparency adjustment results, a dynamic correction mechanism is introduced during subsequent blind zone identification. When smoothing the initial boundary contour, the smoothing parameters are fine-tuned according to the transparency adjustment range to improve boundary contour accuracy. The corrected blind zone boundary contour is compared with the historical blind zone boundary contours stored in the database, and a shape context matching algorithm is used to calculate the similarity value. If the similarity exceeds a preset similarity threshold, it indicates that the current blind zone is highly similar to the historical blind zone, triggering a parameter update operation for the blind zone prediction model. This blind zone prediction model adopts a stacked long short-term memory network architecture. The specific architecture consists of: an input layer that receives the time series and location series data of historical blind zone occurrences; followed by two long short-term memory network hidden layers, each with 256 hidden units, with a random dropout layer with a dropout rate of 0.3 added between layers to prevent overfitting; the output of the last long short-term memory network layer is connected to a fully connected layer with 128 neurons; the output layer is a fully connected layer, with the number of neurons matching the dimension of the prediction target, i.e., the coordinates and probability of the possible future blind zone locations. The Long Short-Term Memory (LSTM) network layer controls the memory and forgetting of temporal information through forgetting, input, and output gates, enabling it to capture long-term dependencies. The random dropout layer randomly ignores some neurons during training, enhancing the model's generalization ability. The fully connected layer maps features to the prediction target. The loss function uses a weighted sum of mean squared error and binary cross-entropy, simultaneously optimizing both location prediction accuracy and probability. The model is trained using blind zone boundary contour sequences recorded in historical interaction sessions, with a training dataset of 50,000 sample pairs. The input consists of timestamps and location coordinates of blind zone occurrences in the past 10 interaction sessions, and the output is the probability distribution of possible blind zone locations in the next two interaction sessions. The Adam optimizer is used, with an initial learning rate of 0.0005, a batch size of 32, and 100 training epochs. The average intersection-over-union (IoU) ratio is used as the evaluation metric.

[0087] For example, in step S310: after generating the spatial emotion distribution map of the fusion operation intention and voice emotion, the spatial emotion distribution map is processed by calculating the map entropy value to obtain the overall information entropy value of the spatial emotion distribution map. The difference between the overall information entropy value and the preset benchmark information entropy value is calculated to obtain the information entropy change. The information entropy change is used to characterize the degree of fluctuation of the user's emotional complexity during the interaction process.

[0088] After generating the spatial sentiment distribution map, a global quantitative analysis is performed. A grayscale histogram is generated by statistically analyzing the distribution of grayscale values ​​of all pixels in the spatial sentiment distribution map. The probability of each grayscale level is calculated based on the histogram. The Shannon entropy formula is applied to multiply the probability of each grayscale level by its logarithm, and the sum of these probabilities over all grayscale levels is taken as the negative value to obtain the overall information entropy value. A preset baseline information entropy value is obtained, and the current overall information entropy value is subtracted from the baseline information entropy value to obtain the change in information entropy. This change in information entropy characterizes the degree of fluctuation in the emotional complexity of the user during the interaction.

[0089] Step S320: Based on the positive and negative directions and absolute value of the change in information entropy, determine whether the user's emotional state during the interaction process tends to become more complex or more simple.

[0090] The change in information entropy is assessed by determining its sign and magnitude. If the change in information entropy is positive and its absolute value is large, the user's emotional state is considered to be becoming more complex; if the change in information entropy is negative and its absolute value is large, the user's emotional state is considered to be becoming simpler; if the change in information entropy is close to zero, the user's emotional state is considered to be stable.

[0091] Step S330: If the change in information entropy is positive and its absolute value exceeds a preset complexity threshold, it is determined that the user's emotional state is becoming more complex, and a local analysis operation on the spatial emotional distribution map is triggered. In the local analysis operation, the standard deviation of the color channel values ​​of pixels in each predetermined local area of ​​the spatial emotional distribution map is calculated, and the local area with the largest standard deviation value is taken as the key analysis area. Pixel-level analysis is performed on the key analysis area, and the grayscale value change rate and color channel value change rate of each pixel in the key analysis area are extracted.

[0092] If the change in information entropy is positive and its absolute value exceeds a preset complexity threshold, it confirms that the user's emotional state has become significantly more complex, triggering a local analysis operation. The spatial emotional distribution map is divided into multiple predetermined local regions according to a preset window size. For each local region, the standard deviation of the red, green, and blue channels for all pixels is calculated, and the local region with the largest standard deviation is identified as the key analysis region. Pixel-level analysis is performed on this key analysis region. For each pixel within the key analysis region, its grayscale value is differiated from the grayscale values ​​of its neighboring pixels to obtain the grayscale value change rate. The spatial gradient of the red, green, and blue channel values ​​is calculated, and the root mean square is taken as the color channel value change rate.

[0093] Step S340: Calculate the correlation between the grayscale value change rate and the color channel value change rate to obtain the correlation coefficient between emotion and operation. If the absolute value of the correlation coefficient between emotion and operation is lower than a preset correlation threshold, it is determined that there is a decoupling phenomenon between emotion and operation in the key analysis area. The key analysis area with the decoupling phenomenon between emotion and operation is marked as a potential interactive experience problem area, and the location coordinates of the potential interactive experience problem area are added to the list of areas to be optimized.

[0094] The grayscale value change rate sequence and color channel value change rate sequence of all pixels in the key analysis area are used as two variables for correlation analysis. The Pearson correlation coefficient method is used to calculate the mean, covariance, and standard deviation of the two sequences respectively. The covariance is divided by the product of the two standard deviations to obtain the correlation coefficient between sentiment and operation. A preset correlation threshold is set. If the absolute value of the correlation coefficient is lower than the threshold, it is determined that there is a decoupling phenomenon between sentiment and operation in the area. The area is marked as a potential interactive experience problem area, and its location coordinates are added to the list of areas to be optimized.

[0095] Step S350: Merge the list of areas to be optimized with the location coordinates of the negative emotional experience blind spot to generate a comprehensive optimization area set, which is used to guide the generation of subsequent interface element enhancement prompts.

[0096] The list of areas to be optimized is merged with the location coordinates of blind spots in negative emotional experiences. Areas that overlap or are adjacent in space are then fused to generate a larger comprehensive area, ultimately resulting in a set of comprehensive optimized areas. This set of comprehensive optimized areas is then used as input when generating enhanced prompts for interface elements, ensuring that the generated prompts cover a wider range of potential experience problems.

[0097] Step S410: After generating the interface element enhancement prompt instruction, while sending the interface element enhancement prompt instruction to the interactive terminal, simultaneously record the sending timestamp and instruction content summary of the interface element enhancement prompt instruction.

[0098] While sending the UI element enhancement prompt command to the UI rendering engine, a recording operation is triggered, creating a new record entry in the log database, recording the current high-precision time as the sending timestamp, extracting a summary of the complete command content, extracting the list of UI element identifiers and main visual enhancement parameters contained in the command, organizing them into a structured command content summary and storing it in association with the sending timestamp.

[0099] Step S420: After the interactive terminal completes the visual protrusion display operation triggered by the interface element enhancement prompt instruction, the user interaction feedback data after execution is collected. The user interaction feedback data includes the update record of the user's touch pressure distribution trajectory on the new interface and the update record of the change in eye gaze heatmap.

[0100] After the interface rendering engine completes the visual protrusion display operation, a new round of data collection is restarted. The collection process is exactly the same as step S110. The collection object is the user's interactive behavior on the enhanced new interface, and the updated records of touch pressure distribution trajectory and eye gaze heatmap changes are obtained, which constitute the user interaction feedback data after execution.

[0101] Step S430: Perform real-time sentiment analysis on the user interaction feedback data to generate an optimized user sentiment state assessment value. Compare the optimized user sentiment state assessment value with the sentiment state assessment value before the interface element enhancement prompt instruction is sent, and calculate the degree of improvement in sentiment state.

[0102] The user interaction feedback data is processed using the same sentiment analysis process as in step S130 to obtain an optimized user sentiment state assessment value. The sentiment state assessment value of the interaction process before reinforcement is retrieved from memory or logs. The optimized assessment value is compared with the pre-reinforcement assessment value. If it is a discrete sentiment category label, it is compared whether it has changed from negative to non-negative. If it is a continuous value, the difference between the optimized value and the pre-optimization value is calculated; this difference represents the degree of improvement in sentiment state.

[0103] Step S440: If the improvement in the emotional state exceeds a preset effective improvement threshold, then the enhancement prompt strategy corresponding to the interface element enhancement prompt instruction is marked as an effective strategy; if the improvement in the emotional state is lower than the effective improvement threshold, then the enhancement prompt strategy corresponding to the interface element enhancement prompt instruction is marked as an invalid strategy.

[0104] A preset effective improvement threshold is set. For discrete emotion tags, this threshold can be a logical condition, such as changing from negative to non-negative; for continuous values, this threshold is a specific numerical value. The magnitude of emotion state improvement is compared with the preset effective improvement threshold. If the improvement exceeds the threshold, the enhancement prompt strategy represented by the currently executed interface element enhancement prompt instruction is marked as an effective strategy; if the improvement is below the threshold, it is marked as an invalid strategy.

[0105] Step S450: Based on the set of visual salient display parameters used by the reinforcement prompt strategy marked as an effective strategy, adjust the weights of the corresponding visual salient display parameter combinations in the preset function reinforcement effect library, increasing the weight values ​​of the parameters corresponding to the effective strategy; based on the feature parameters of the reinforcement prompt strategy marked as an invalid strategy, adjust the weights of the set of visual salient display parameters in the preset function reinforcement effect library, decreasing the weight values ​​of the parameters corresponding to the invalid strategy; normalize the adjusted weight values, and re-store the updated set of visual salient display parameters and their corresponding weight values ​​in the preset function reinforcement effect library.

[0106] In the functional enhancement effect library, each set of visual salience display parameters is associated with a weight value representing its selection priority. For the parameter set corresponding to an effective strategy, its weight value is increased by a preset increment; for the parameter set corresponding to an ineffective strategy, its weight value is decreased by a preset decrement. After adjustment, all weight values ​​are added together to obtain a sum, and each weight value is divided by the sum to obtain a new normalized weight value. The visual salience display parameter set with updated weights and its new weight values ​​are then written back into the functional enhancement effect library, completing one iteration of reinforcement learning-based effect library optimization.

[0107] Step S510: During the process of generating the operation intention focus area distribution map, the time series of pressure sensing values ​​of each touch point in the interface touch pressure distribution trajectory is extracted simultaneously, and the time series of pressure sensing values ​​is processed by frequency domain transformation to obtain the pressure frequency domain feature spectrum.

[0108] In parallel with the generation of the operation intent focused area distribution map, another analysis is performed to extract the pressure sensing values ​​of all touch points from the interface pressure distribution trajectory. These values ​​are then arranged into a one-dimensional time series, strictly according to timestamp order, i.e., the time series of pressure sensing values. A Fast Fourier Transform algorithm is applied to this one-dimensional time series to convert it from a time-domain signal to a frequency-domain signal, obtaining a pressure frequency-domain feature spectrum composed of amplitude components at a series of different frequencies.

[0109] Step S520: Identify the dominant frequency component and the secondary frequency component of pressure fluctuation from the pressure frequency domain feature spectrum, and record the frequency value and amplitude of the dominant frequency component and the secondary frequency component of pressure fluctuation. Match the frequency value of the dominant frequency component of pressure fluctuation with a preset tense operation frequency range. If the frequency value of the dominant frequency component of pressure fluctuation falls into the tense operation frequency range, determine that the user's current operation state is a tense operation state, and extract the eye gaze point distribution data of the same period as the tense operation state from the eye gaze heatmap change record.

[0110] The pressure frequency domain feature spectrum is analyzed to identify the frequency component with the largest amplitude as the dominant frequency component of pressure fluctuation, and its frequency and amplitude are recorded. The frequency component with the second largest amplitude is identified as the secondary frequency component of pressure fluctuation, and its frequency and amplitude are also recorded. A preset tension operation frequency range is set, which is a frequency interval used to characterize the typical fluctuation range of the user's pressing frequency under tension. The frequency value of the dominant frequency component of pressure fluctuation is numerically matched with this range. If the frequency value is greater than or equal to the lower limit and less than or equal to the upper limit, the user's current operation state is determined to be a tension operation state. Based on the time interval that triggered the determination, the eye gaze heatmap change records are traced back, and all time frames with timestamps falling within this interval are selected. The coordinates of all eye gaze points are extracted to form an eye gaze point distribution dataset corresponding to the tension operation period.

[0111] Step S530: Perform gaze point density clustering analysis on the gaze point distribution data during the same period as the tense operation state to generate the coordinates of the gaze point cluster center under the tense state; identify the region whose fusion intention density value exceeds the preset high density threshold from the operation intention focus area distribution map, and calculate the center coordinates of the region as the high density region center coordinates; calculate the offset between the gaze point cluster center coordinates and the high density region center coordinates to obtain the gaze operation offset vector.

[0112] A density-based spatial clustering algorithm is used to process the eye gaze point distribution dataset. Regions with sufficiently high density and interconnectedness are identified as clusters. For each cluster, the average coordinates of all gaze points within it are calculated to obtain the geometric center of the cluster, i.e., the gaze point aggregation center coordinates under tension. A preset high-density threshold is set from the operation intention focus area distribution map. Pixels with grayscale values ​​greater than this threshold are identified as high-density regions. The average row and column indices of all pixels within this region are calculated to obtain the center coordinates of the high-density region. A vector subtraction operation is performed between the gaze point aggregation center coordinates and the high-density region center coordinates. Subtracting the x-coordinate of the high-density region center from the x-coordinate of the gaze point aggregation center yields the x-coordinate offset, and subtracting the y-coordinate from the y-coordinate yields the y-coordinate offset. The two-dimensional vector formed by the x-coordinate offset and y-coordinate offset is the gaze operation offset vector.

[0113] Step S540: If the magnitude of the gaze operation offset vector exceeds the preset offset tolerance threshold, it is determined that the user has an abnormal interaction state where the operation and gaze are inconsistent. The occurrence time and location coordinates of the abnormal interaction state are recorded in the abnormal event log. Based on the frequency of the abnormal interaction state recorded in the abnormal event log, the preset density threshold and the preset overlap ratio threshold are dynamically adjusted. The adjusted preset density threshold and the preset overlap ratio threshold are applied to the subsequent process of identifying the blind spot of negative emotional experience and the process of generating interface element enhancement prompts.

[0114] Calculate the magnitude of the gaze offset vector, which is equal to the square root of the sum of the squares of the horizontal and vertical offsets. Set a preset offset tolerance threshold and compare the magnitude with this threshold. If the magnitude is greater than the threshold, an abnormal interaction state is determined, indicating inconsistency between the user's action and gaze. Record the occurrence time and location coordinates of this abnormal state in the abnormal event log. Periodically count the total number of abnormal interaction states in the abnormal event log, i.e., the occurrence frequency. Dynamically adjust the preset density threshold and preset overlap ratio threshold based on this frequency. When the abnormal frequency increases, appropriately reduce the preset density threshold and preset overlap ratio threshold to improve sensitivity to potential problems; conversely, revert to the previous threshold. Update the adjusted thresholds to the method configuration parameters and apply them to the subsequent process of identifying negative emotional experience blind spots and generating interface element enhancement prompts.

[0115] Step S610: After generating the spatial emotion distribution map of the fusion operation intention and voice emotion, a sliding window analysis is performed on the spatial emotion distribution map on the time axis. The emotion color distribution histogram in each sliding window is extracted, and the similarity is calculated with the emotion color distribution histogram in the adjacent sliding window to obtain the temporal similarity sequence of emotion color distribution. The time point where the similarity value is lower than the preset sudden drop threshold is identified from the temporal similarity sequence of emotion color distribution, and the time point is marked as the emotion turning point.

[0116] After generating the spatial sentiment distribution map, a dynamic analysis is performed on the temporal dimension of the map. A fixed-length time window and a fixed sliding step size are set. Starting from the beginning of the map's timeline, the time window moves with the sliding step size, extracting a corresponding segment of the map for each time window. The color channel values ​​of all pixels within the segment are statistically analyzed to generate a histogram of sentiment color distribution. The similarity between each histogram and the next sliding window histogram is calculated using Bach distance as the similarity metric, resulting in a series of similarity values ​​arranged chronologically, forming a temporal similarity sequence of sentiment color distribution. A preset drop threshold is set, and each similarity value in the temporal similarity sequence is compared to this threshold. When a similarity value falls below the threshold, the time point corresponding to that similarity value—the boundary between two adjacent sliding windows—is marked as a sentiment turning point.

[0117] Step S620: Backtrack the original behavioral data stream within the preset time window before the emotional turning point, analyze the trend of changes in the user interface touch pressure distribution trajectory and the trend of changes in the eye gaze heatmap before the occurrence of the emotional turning point, and construct an emotional turning point precursor feature vector based on the trend of changes in the user interface touch pressure distribution trajectory and the trend of changes in the eye gaze heatmap.

[0118] For each emotional turning point, a preset time window is determined before that point. The interface touch pressure distribution trajectory and eye gaze heatmap changes within this window are extracted from the raw behavioral data stream. The interface touch pressure distribution trajectory is analyzed to determine the trend of pressure sensitivity values ​​over time, using quantitative numerical features such as the slope of the linear fit of pressure sensitivity values ​​within the window to represent this trend. The eye gaze heatmap changes are analyzed to determine the trend of the dispersion of gaze point distribution, using the rate of change of gaze point distribution variance to represent this dispersion. These quantified features are combined into a multi-dimensional feature vector, i.e., the emotional turning point precursor feature vector.

[0119] Step S630: Input the feature vector of the precursor of the emotional turning point into the pre-trained emotional turning point prediction model to obtain the predicted probability value of the emotional turning point occurring within the future time window. If the predicted probability value exceeds the preset prediction trigger threshold, mark the predicted emotional turning point related area on the operation intention focus area distribution map in advance.

[0120] A pre-trained emotional turning point prediction model was constructed, employing a Long Short-Term Memory (LSTM) network architecture. The input layer dimension matched the dimension of the emotional turning point precursor feature vectors. The model contained multiple LSM layers for learning time-series dependencies, with the output of the last LSM layer connected to a fully connected layer. Finally, an output layer with an S-shaped growth curve mapped the fully connected layer's output value to a probability value between 0 and 1. The emotional turning point precursor feature vectors were organized chronologically into a sequence input to the model. Forward propagation was used to calculate the predicted probability of emotional turning points occurring within future time windows from the output layer. A preset prediction trigger threshold was set, and the predicted probability value was compared with this threshold. If the probability value exceeded the threshold, a pre-labeling operation was triggered. The original behavioral data used to generate the precursor feature vectors was backtracked, and the region with the highest concentration of gaze points within the backtracking window was identified. The corresponding location of this region on the operation intent focus area distribution map was labeled as the predicted emotional turning point associated region.

[0121] Step S640: Based on the predicted location coordinates of the emotional transition associated region, generate a tentative interface element enhancement prompt instruction in advance, and cache the tentative interface element enhancement prompt instruction in the instruction cache queue.

[0122] Based on the predicted coordinates of the emotional shift associated region, the same instruction generation logic as in step S150 is invoked to pre-generate interface element enhancement prompts for the interface elements within that associated region. These interface element enhancement prompts are marked as tentative and are not immediately sent to the interface rendering engine for execution. Instead, they are stored in the instruction cache queue, and their corresponding prediction time window information is recorded.

[0123] Step S650: When the actual emotional turning point is confirmed to have occurred, retrieve the corresponding tentative interface element enhancement prompt instruction from the instruction cache queue, convert it into a formal instruction and send it to the interactive terminal. If the actual emotional turning point does not occur within the prediction time window, clear the corresponding tentative interface element enhancement prompt instruction from the instruction cache queue and adjust the internal parameters of the emotional turning point prediction model to reduce the false alarm rate.

[0124] In subsequent real-time processing, the actual emotional turning point is continuously monitored through step S610. When an emotional turning point is confirmed at a certain time, the instruction cache queue is immediately checked to see if there is a pre-cached tentative instruction that matches the time window of the turning point. If found, the status of the instruction is changed from tentative to formal, and it is sent to the interface rendering engine via inter-process communication to trigger a visual highlighting operation. Monitoring continues until the prediction time window has completely ended. If no actual emotional turning point is detected within the entire window, the prediction is considered a false alarm, and the corresponding tentative instruction in the instruction cache queue is cleared. At the same time, the online learning process of the model is triggered, and the feature vector of the premonitory emotional turning point that caused the false alarm is fed back as a negative sample to the emotional turning point prediction model. The emotional turning point prediction model fine-tunes the weight parameters of each layer of its internal long short-term memory network through the backpropagation algorithm to reduce the possibility of similar false alarms in the future.

[0125] For example, in step S710: after generating the spatial emotion distribution map of the fused operation intention and voice emotion, the spatial emotion distribution map is subjected to map topology analysis processing to extract the adjacency relationship between the areas covered by different voice emotion color markers and the density gradient contour distribution pattern formed by different operation intention density values ​​in the spatial emotion distribution map.

[0126] After generating the spatial sentiment distribution map, its topological structure is analyzed. An image segmentation algorithm is used to segment continuous regions in the map composed of the same or similar color channel values ​​into different sentiment color regions. The boundaries of each region are recorded, and the adjacency relationships between different regions are analyzed. Simultaneously, the map is treated as a two-dimensional grayscale field, and the grayscale gradient of each pixel—the rate of change of grayscale value in the horizontal and vertical directions—is calculated. Density gradient contour lines are drawn by tracing points with equal grayscale values.

[0127] Step S720: Based on the adjacency relationship between the areas covered by different voice emotion color markers, construct an emotion color region adjacency graph. The nodes in the emotion color region adjacency graph correspond to each continuous monochrome emotion region. The edges in the emotion color region adjacency graph connect two adjacent emotion regions of different colors, and mark the boundary length of the two emotion regions and the average difference in gray values ​​on both sides of the boundary on the edges.

[0128] Each continuous monochromatic emotion region is abstracted as a node, with node attributes including the emotion category and area of ​​the region. For each pair of adjacent emotion regions of different colors, an edge is created between the two corresponding nodes. The length of the common boundary between the two regions is calculated as the weight attribute of the edge. At the same time, the pixels on both sides of the boundary are traversed, and the average gray value of the pixels in the regions on both sides of the boundary is calculated. The absolute value of the difference between the two average values ​​is used as another attribute of the edge, namely the average difference of gray values ​​on both sides of the boundary. This constructs an adjacency graph of emotion color regions with attributes.

[0129] Step S730: Based on the distribution pattern of density gradient contour lines formed by different operational intent density values, extract the closed loop structure of the density gradient contour lines and the spatial distribution characteristics of the density gradient contour lines. The closed loop structure of the density gradient contour lines is used to indicate the local peak region of the operational intent density. The spatial distribution characteristics of the density gradient contour lines characterize the rate of change of the operational intent density in space by calculating the number of contour lines per unit area.

[0130] Further analysis of the density gradient contour lines identifies all contour lines forming closed loops, each indicating a local peak or trough region of the operational intent density. The entire map is divided into multiple sub-regions. The total length of contour lines within each sub-region is calculated and divided by the area of ​​that sub-region to obtain the number of contour lines per unit area, which characterizes the rate of change of operational intent density in space within the sub-region.

[0131] Step S740: Perform spatial overlay analysis on the adjacency map of the emotional color region and the distribution pattern of the density gradient contour lines to identify positive emotional color mark island regions that are surrounded or partially surrounded by negative emotional color mark regions. The positive emotional color mark island region refers to a region that has positive emotional color marks, and the area ratio of negative emotional color mark regions in its surrounding adjacent regions exceeds a preset ratio threshold.

[0132] The adjacency graph of the emotional color region and the distribution pattern of density gradient contour lines are spatially superimposed in a unified coordinate system. All nodes representing positive emotional categories are traversed. For each positive emotional node, all its adjacent nodes connected by edges in the adjacency graph are checked. The total boundary length of the nodes representing negative emotional categories among the adjacent nodes is counted. The proportion of the boundary length of the negative emotional adjacent region to the total boundary length of all adjacent regions around the positive emotional node is calculated. If this proportion exceeds a preset threshold, the positive emotional region is identified as an isolated region with positive speech emotional color labeling.

[0133] Step S750: Mark the isolated areas of the positive voice emotion color label as emotion resistance areas. The emotion resistance area represents a local interface area where the user still maintains positive emotions in an overall negative emotional atmosphere, and record the position coordinates and boundary contour of the emotion resistance area in the spatial emotion distribution map.

[0134] Each isolated region marked with positive voice emotion is assigned a new label, namely an emotion resistance region. This label reveals the special property of this region that can still evoke or maintain positive emotions in the user in the surrounding negative emotional environment. The precise boundary contour of this emotion resistance region in the spatial emotion distribution map is recorded and stored in association with the label.

[0135] Step S760: Perform density gradient penetration analysis on the emotional resistance region, calculate the number of density gradient contour lines penetrating the boundary of the emotional resistance region, and the gradient direction change angle of each penetrating density gradient contour line when it passes through the boundary of the emotional resistance region.

[0136] For each emotional resistance region, a penetration analysis is performed, and the number of contour lines crossing the boundary of the region is counted using density gradient contour lines. For each contour line that crosses the boundary, the angle between the tangent direction of the contour line and the normal direction of the boundary is calculated at the intersection point of the contour line and the boundary. The size of this angle represents the angle of gradient direction change when the contour line crosses the boundary.

[0137] Step S770: If the number of penetrations of the density gradient contour lines exceeds a preset penetration threshold and the average value of the gradient direction change angle is less than a preset angle change threshold, then the emotional resistance region is determined to have emotional radiation resistance capability. The emotional radiation resistance capability indicates that the region can resist the spread and intrusion of surrounding negative emotions.

[0138] Set preset penetration quantity thresholds and preset angle change thresholds. For each emotional resistance region, compare its contour penetration quantity with the penetration quantity threshold, and compare the average angle change of all gradient directions with the angle change threshold. If the penetration quantity exceeds the threshold and the average angle change is less than the threshold, it indicates that the boundary of the region exhibits a stable barrier in the density gradient, effectively hindering or altering the intrusion path of external contour lines. This indicates that the emotional resistance region has the ability to resist emotional radiation and can resist the spread and intrusion of surrounding negative emotions.

[0139] Step S780: Extract the set of interface element identifiers within the emotional resistance region that possesses emotional radiation resistance capabilities, and input the set of interface element identifiers into a pre-trained interface element feature analysis model to obtain common visual feature parameters and common layout feature parameters of the corresponding interface elements in resisting the spread of negative emotions. Based on the common visual feature parameters and the common layout feature parameters, generate interface element emotional resistance design specifications. The interface element emotional resistance design specifications include recommended color combinations, recommended element shape types, and recommended element spacing ranges.

[0140] For emotionally resistant regions with resistance to emotional radiation, the interface layout description file is used to identify all interface elements whose bounding box coordinates fall within this region, and the element identifiers of these interface elements are extracted to form a set. This set of element identifiers is then input into a pre-trained interface element feature analysis model, which employs an architecture combining graph neural networks and convolutional neural networks. The specific architecture consists of the following components: an input layer receives a set of interface element identifiers, and an embedding layer maps each element identifier to a 128-dimensional embedding vector; the graph neural network part contains three graph convolutional layers, each layer aggregating the feature information of adjacent nodes, with the adjacency relationship between nodes defined by the spatial proximity in the interface layout; a global average pooling layer is connected after the graph convolutional layers to aggregate the features of all nodes; simultaneously, the visual features of each interface element are extracted through a lightweight convolutional neural network, which contains two convolutional layers and two pooling layers, outputting a 64-dimensional visual feature vector; the layout features output by the graph neural network are concatenated with the visual features output by the convolutional neural network along the feature dimension to obtain a 192-dimensional comprehensive feature vector; finally, two fully connected layers are connected, with 96 and 48 neurons respectively, and the output layer outputs common visual feature parameters and common layout feature parameters according to task requirements. The graph convolutional layer uses the Laplacian matrix to weight and aggregate the features of neighboring nodes, thus achieving information propagation in the graph structure; the convolutional neural network part uses a 3x3 convolutional kernel to extract the local visual patterns of the element screenshot area; feature concatenation realizes the fusion of information from different modalities; the loss function adopts multi-task learning loss, which includes a weighted sum of visual feature reconstruction loss and layout feature prediction loss.

[0141] The interface element feature analysis model extracts visual features such as color histograms, shape descriptors, and layout features such as spacing with adjacent elements and absolute screen position for each element based on its identifier. Through pooling operations, it analyzes the commonalities of all elements in the set, outputting quantified common visual feature parameters such as the dominant color range, shape types primarily using rounded rectangles, and common layout feature parameters such as the average spacing between elements and a preference for positions near screen edges. Based on these parameters, it generates an interface element emotional resistance design specification, including recommended color combinations using the dominant color and its complementary color as recommended schemes, recommended element shape types as rounded rectangles with recommended rounded corner radius ranges, and recommended element spacing ranges.

[0142] Step S790: Encapsulate the interface element emotional resistance design specifications into interface design auxiliary instructions, and send the interface design auxiliary instructions to the interface design tool of the smart terminal to trigger real-time optimization suggestions for interface design parameters.

[0143] The emotional resistance design guidelines for interface elements are formatted and encapsulated to generate interface design auxiliary instructions. These instructions include an instruction header, the specification body, and a checksum. These instructions are then sent to an interface design tool running on a central control terminal via an application programming interface (API). Upon receiving the instructions, the design tool parses the design guidelines and provides real-time optimization suggestions to interface designers through floating prompts and highlighted parameter presets. This process feeds back the emotional resistance patterns discovered in data analysis into the future interface design workflow.

[0144] For example, in step S810: before generating the interface element enhancement prompt instruction for the blind spot of negative emotional experience, all regions with negative voice emotion color markings are extracted from the spatial emotion distribution map, and the regions are arranged in descending order of area to generate a negative emotion region area sorting list.

[0145] Before generating interface element enhancement prompts, a global scan of the spatial emotion distribution map is performed. Based on the emotion color and emotion category comparison table, the connected regions formed by all pixels with negative voice emotion color markers in the map are identified. The area of ​​each region, i.e. the total number of pixels contained therein, is calculated. The region information is arranged into a negative emotion region area sorting list in descending order of area.

[0146] Step S820: Based on the preset source region quantity threshold N, select the top N negative emotion regions with the largest areas from the negative emotion region area sorting list as the main negative emotion source regions, and record the geometric center coordinates and region boundary coordinates of each main negative emotion source region.

[0147] A preset threshold N for the number of source regions is set. The top N regions (i.e., the N largest negative emotion regions) are selected from the list of negative emotion regions sorted by area, and these are defined as the main negative emotion source regions, which are considered the main origins of negative emotion. For each main negative emotion source region, its geometric center coordinates are calculated, which is the average of the row and column indices of all the pixels it contains. At the same time, its precise region boundary coordinates are recorded again.

[0148] Step S830: Perform emotional diffusion direction analysis on each major negative emotional source region, extract the speech emotional color mark gradient pattern of the surrounding area of ​​the major negative emotional source region, and determine the dominant emotional diffusion direction vector of the major negative emotional source region based on the speech emotional color mark gradient pattern.

[0149] For each major negative emotion source region, the annular neighborhood outside its boundary is analyzed to observe the changing patterns of speech emotion coloring within this region. Starting from the boundary of the source region and moving outward along different radial directions, the rate at which the emotion category changes from negative to neutral or positive is statistically analyzed. The direction with the slowest rate of emotion category change and the farthest extension of negative emotion is determined as the dominant emotion diffusion direction of that source region, and this direction is represented as a two-dimensional vector, namely the dominant emotion diffusion direction vector.

[0150] Step S840: Based on the geometric center coordinates of each major negative emotion source region and the dominant direction vector of emotion diffusion, draw an emotion diffusion ray on the spatial emotion distribution map. The emotion diffusion ray starts from the geometric center coordinates and extends along the direction of the dominant direction vector of emotion diffusion until it encounters a position where the voice emotion color mark changes in the opposite direction.

[0151] Starting from the geometric center coordinates of each major negative emotional source region, a ray is drawn along the dominant direction vector of emotional diffusion in that source region. The ray extends outward continuously, and the emotional category of the pixels that the ray passes through is monitored in real time. When the ray encounters a pixel whose emotional category is no longer negative, that position is defined as the end point of the ray. The line segment from the starting point to the end point is the emotional diffusion ray of that major negative emotional source region.

[0152] Step S850: The strip-shaped area covered by the multiple emotional diffusion rays during their extension is defined as the emotional diffusion influence corridor, and the interface area within the emotional diffusion influence corridor is affected by the negative emotional radiation of the main negative emotional source area.

[0153] For each major negative emotion source region, a preset width is set with its emotion diffusion ray as the central axis. The strip area formed by all pixels within half the width on both sides of the central axis is determined as the emotion diffusion influence corridor of the source region. This emotion diffusion influence corridor represents the main path and influence range of negative emotions radiating and spreading outward from the source region. When the diffusion rays of multiple source regions converge or overlap in space, they together form a larger emotion diffusion influence corridor.

[0154] Step S860: Overlay the emotional diffusion influence corridor with the operation intention focus area distribution map to identify the interface area located in the emotional diffusion influence corridor whose operation intention density value is lower than the preset mobility threshold in a number of consecutive time frames exceeding the preset time period, and mark the interface area as the emotional blocking area.

[0155] The distribution map of the corridor affected by the spread of emotions and the area focused on the operational intent is spatially overlaid to identify the screen pixel grids within the corridor coverage area. The data of multiple consecutive time frames on which the fusion intent density value of these grids was generated is traced back. For each candidate grid, it is checked whether its fusion intent density value is always lower than a preset mobility threshold in a number of consecutive time frames exceeding a preset time period. If the condition is met, the continuous area formed by the above grids is marked as an emotional blockage area.

[0156] Step S870: Perform blockage cause analysis on the emotional blockage area, extract the operation intention density value change curve of the emotional blockage area in continuous time frames, and identify the duration segment from the operation intention density value change curve where the density value is continuously lower than the preset blockage density threshold and the duration exceeds the preset time period threshold. Calculate the proportion of the number of times the trajectory pattern of the touch point appears in the duration segment to the total number of touch points in the time period, and use it as the emotional blockage severity index of the emotional blockage area. The area where the emotional blockage severity index exceeds the preset blockage threshold is identified as the key area for emotional guidance.

[0157] For each emotional blockage area, the curve showing the change of the average fusion intent density value over time is extracted from the corresponding time frame data; this is the operation intent density value change curve. A preset blockage density threshold, stricter than the liquidity threshold, is set. Time periods where the density value consistently falls below this threshold across consecutive time periods are identified on the operation intent density value change curve. Severe blockage periods exceeding the preset time period threshold are then selected. For each severe blockage period, the trajectories of all touch points within that period are analyzed. Abnormal trajectory patterns are defined as rapid, repeated clicks within a small area or prolonged touches and swipes without lifting the touch button. The number of touch points exhibiting abnormal trajectory patterns is divided by the total number of touch points within that time period to obtain the ratio, which is the emotional blockage severity index. This emotional blockage severity index is compared with the preset blockage threshold. If the index exceeds the preset threshold, the area is designated as a key area for emotional guidance.

[0158] Step S880: For the key area of ​​emotional guidance, generate an interface emotional guidance instruction that includes interface element rearrangement parameters and interface element functional flow replanning parameters, and send the interface emotional guidance instruction to the interface rendering engine of the smart terminal to trigger the dynamic rearrangement of interface elements and the visual guidance operation of functional flow.

[0159] For each key area for emotional guidance, an interface emotional guidance instruction is generated, containing parameters for rearranging interface elements and replanning the functional flow of interface elements. The rearrangement parameters specify moving certain buttons within this area to a more central position on the screen or increasing the spacing between them and other elements. The functional flow replanning parameters specify drawing dynamic arrow guide lines on the interface to guide the user's eye from one operation point to the next most likely operation point. The generated interface emotional guidance instruction is sent to the central control terminal's interface rendering engine, which parses and executes the corresponding dynamic rearrangement and visual guidance operations.

[0160] Figure 2 This application illustrates an intelligent interactive experience optimization system 100 incorporating affective computing, comprising a processor 1001, a memory 1003, and program code stored in the memory 1003. The processor 1001 executes the program code to implement the steps of the intelligent interactive experience optimization method incorporating affective computing. The processor 1001 and the memory 1003 are connected, for example, via a bus 1002. Optionally, the intelligent interactive experience optimization system 100 may further include a transceiver 1004, which can be used for data interaction between this intelligent interactive experience optimization system and other intelligent interactive experience optimization systems incorporating affective computing, such as sending and / or receiving data. It should be noted that in actual scheduling, the transceiver 1004 is not limited to one, and the structure of this intelligent interactive experience optimization system 100 incorporating affective computing does not constitute a limitation on the embodiments of this application. The memory 1003 is used to store the program code executing the embodiments of this application, and its execution is controlled by the processor 1001. The processor 1001 is used to execute program code stored in the memory 1003 to implement the steps shown in the foregoing method embodiments.

[0161] This application provides a computer-readable storage medium storing program code, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0162] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A method for optimizing intelligent interactive experiences by incorporating affective computing, characterized in that, The method includes: The raw behavioral data stream generated synchronously by the user during multi-round task dialogue with the smart terminal is collected. The raw behavioral data stream includes the interface touch pressure distribution trajectory, the voice command spectral feature sequence, and the eye gaze heatmap change record. The interface touch pressure distribution trajectory and the eye gaze heatmap change record are subjected to dual-modal fusion encoding processing to generate a distribution map of the user's operation intention focus area on the interface; Based on the spectral feature sequence of the voice command, a voice emotion color mark is superimposed on the distribution map of the operation intention focus area to obtain a spatial emotion distribution map that integrates operation intention and voice emotion. The interface regions with negative voice emotion markers and operation intention density below a preset threshold are identified from the spatial emotion distribution map and marked as negative emotion experience blind spots. Based on the location coordinates of the negative emotional experience blind spot in the spatial emotional distribution map, an interface element enhancement prompt instruction is generated for the negative emotional experience blind spot, and the interface element enhancement prompt instruction is sent to the interactive terminal to trigger the visual protrusion display operation of the interface element in the blind spot.

2. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The process of performing dual-modal fusion encoding on the interface touch pressure distribution trajectory and the eye gaze heatmap change record to generate a user's operation intent focus area distribution map on the interface includes: The interface touch pressure distribution trajectory is parsed from the original behavior data stream. The interface touch pressure distribution trajectory consists of multiple touch points, each of which includes the screen horizontal coordinate, screen vertical coordinate, pressure sensing value and timestamp. All touch points in the interface touch pressure distribution trajectory are sorted according to their timestamps to generate an ordered touch point sequence. Touch points with pressure sensitivity values ​​greater than the basic pressure threshold are selected from the ordered touch point sequence as valid operation touch points. The number of valid operation touch points falling into each screen pixel grid is counted to obtain the number of valid touch points in each screen pixel grid. The number of valid touch points in each screen pixel grid is divided by the sum of the number of valid touch points in all screen pixel grids to obtain the touch operation probability density value corresponding to each screen pixel grid. Based on the touch operation probability density value corresponding to each screen pixel grid, an initial touch density distribution map is drawn in the screen pixel grid coordinate system. The gray value of each pixel in the initial touch density distribution map is proportional to the touch operation probability density value of the grid in which it belongs. Eye gaze heatmap change records are parsed from the original behavioral data stream. The eye gaze heatmap change records contain a continuous time frame sequence. Each time frame records a set of eye gaze point coordinates and the gaze duration corresponding to each eye gaze point. The eye fixation point coordinates of each time frame are mapped to the screen pixel grid coordinate system. Based on the fixation duration corresponding to each eye fixation point, the fixation duration is accumulated for the screen pixel grid where the eye fixation point is located, generating the cumulative fixation time value of each screen pixel grid. The cumulative fixation time values ​​of all screen pixel grids are normalized to obtain the normalized fixation time density value corresponding to each screen pixel grid. Based on the normalized gaze time density value, an initial gaze density distribution map is drawn in the screen pixel grid coordinate system. The gray value of each pixel in the initial gaze density distribution map is proportional to the normalized gaze time density value of the grid in which it is located. Spatial registration is performed on the initial touch density distribution map and the initial gaze density distribution map to make them use the same screen pixel grid coordinate system and the same grid resolution; The touch operation probability density value of each screen pixel grid in the initial touch density distribution map is weighted and summed with the normalized gaze time density value of the corresponding screen pixel grid in the initial gaze density distribution map to obtain the fusion intent density value of each screen pixel grid. Based on the fusion intent density values ​​of all screen pixel grids, a grayscale distribution map is redrawn in the screen pixel grid coordinate system, and the grayscale distribution map is used as the distribution map of the user's operation intent focus area on the interface.

3. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The step of overlaying voice emotion color markers onto the operation intent focus area distribution map based on the voice command spectral feature sequence to obtain a spatial emotion distribution map that integrates operation intent and voice emotion includes: The speech command spectral feature sequence is parsed from the original behavior data stream. The speech command spectral feature sequence consists of multiple speech frame spectral vectors arranged in chronological order. Each speech frame spectral vector contains the energy amplitude of the speech frame at multiple frequency points. Speech emotion feature extraction is performed on the spectral vector of each speech frame in the speech command spectral feature sequence. The spectral centroid offset, spectral roll-off rate of change, and spectral passband energy distribution ratio of each speech frame spectral vector are calculated and combined to generate a three-dimensional speech emotion feature vector corresponding to each speech frame. This vector is then input into a pre-trained speech emotion classifier to obtain three posterior probability values ​​output by the speech emotion classifier indicating that the speech frame belongs to the positive emotion category, neutral emotion category, and negative emotion category, respectively. The emotion category corresponding to the largest posterior probability value is selected from the three posterior probability values ​​as the speech emotion category label of the speech frame. Based on the temporal order of each speech frame in the speech command spectral feature sequence, a speech frame start timestamp and a speech frame end timestamp are assigned to each speech frame to form a speech emotion category label sequence with time interval information. The speech emotion category label sequence with time interval information is then aligned with the operation intention focus area distribution map on the time axis. The operation intention focus area distribution map has a time axis that is synchronized with the original behavior data stream. Traverse each voice emotion category label in the sequence of voice emotion category labels with time interval information, and extract the fusion intent density value distribution map corresponding to all time points in the time interval from the operation intent focus area distribution map based on the voice frame start timestamp and voice frame end timestamp of each voice emotion category label, and generate a series of time domain sub-distribution maps corresponding to the time interval. Each time-domain sub-distribution map is assigned a speech emotion category label, which is the same as the speech emotion category label in the corresponding time interval of the time-domain sub-distribution map; The voice emotion category markers are superimposed on the corresponding temporal sub-distribution map in the form of a semi-transparent color layer to form a temporal sub-map with emotion color coverage. The temporal sub-maps with emotion color coverage are arranged in chronological order according to their corresponding time intervals, and adjacent temporal sub-maps are smoothly transitioned and merged at the time boundary to eliminate the color abrupt boundary between temporal sub-maps caused by changes in emotion category, thereby generating a continuous emotion color gradient layer. The emotional color gradient layer is overlaid and fused with the complete operation intention focus area distribution map, so that the color information in the emotional color gradient layer is precisely covered on the corresponding time area of ​​the operation intention focus area distribution map according to the time axis, resulting in a two-dimensional spatiotemporal image that contains both operation intention density information and voice emotional color information. The two-dimensional spatiotemporal image is post-processed, and a time scale bar is added to the bottom of the two-dimensional spatiotemporal image, and a legend corresponding to the emotional color and emotional category is added to the right side of the two-dimensional spatiotemporal image, finally generating a spatial emotional distribution map that integrates operation intention and voice emotion.

4. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The step of identifying interface regions with negative voice emotion markers and operation intent density below a preset threshold from the spatial emotion distribution map, and marking these interface regions as negative emotion experience blind spots, includes: The spatial emotion distribution map is analyzed. The spatial emotion distribution map is composed of a pixel matrix. Each pixel stores a gray value and a color channel value. The gray value represents the operation intention density at the location of the pixel, and the color channel value represents the voice emotion color mark superimposed at the location of the pixel. Traverse each pixel in the spatial emotion distribution map, read the color channel value of each pixel, and convert the color channel value of each pixel into the corresponding voice emotion category identifier according to the preset emotion color and emotion category lookup table; Determine whether the voice emotion category identifier corresponding to each pixel is a negative emotion category identifier. If the voice emotion category identifier is a negative emotion category identifier, then mark the pixel as a negative emotion candidate pixel. For each pixel marked as a negative emotion candidate pixel, read its stored gray value and compare the gray value with a preset density threshold to determine whether the gray value is less than the preset density threshold. If the gray value of the negative emotion candidate pixel is less than the preset density threshold, the negative emotion candidate pixel is marked as a blind spot seed pixel, and the pixel row index and pixel column index of the blind spot seed pixel in the spatial emotion distribution map are recorded. Connectivity analysis is performed on all pixels marked as blind spot seed pixels, and adjacent blind spot seed pixels are aggregated into a connected set of pixels. Each connected set of pixels constitutes a candidate blind spot connected region. Calculate the total number of pixels contained in each candidate blind zone connected region, divide the total number of pixels by a preset minimum blind zone area threshold, determine whether the candidate blind zone connected region meets the minimum area requirement, filter out candidate blind zone connected regions that do not meet the minimum area requirement, and for candidate blind zone connected regions that meet the minimum area requirement, extract the minimum pixel row index, maximum pixel row index, minimum pixel column index, and maximum pixel column index of all pixels in the candidate blind zone connected region. Based on the minimum pixel row index, the maximum pixel row index, the minimum pixel column index, and the maximum pixel column index, an outer rectangle is drawn on the spatial sentiment distribution map. The four sides of the outer rectangle correspond to the row containing the minimum pixel row index, the row containing the maximum pixel row index, the column containing the minimum pixel column index, and the column containing the maximum pixel column index, respectively. The rectangular area enclosed by the circumscribed rectangle in the spatial emotion distribution map is determined as the preliminary boundary outline of the negative emotion experience blind zone, and the starting row coordinates, ending row coordinates, starting column coordinates, and ending column coordinates of the negative emotion experience blind zone are recorded according to the preliminary boundary outline. The initial boundary contour is smoothed by replacing the four right angles of the circumscribed rectangle with rounded corners with a certain radius of curvature to generate the final boundary contour of the emotional negative experience blind zone. All pixels within the final boundary contour are uniformly marked as emotional negative experience blind zone pixels, thus completing the marking of the emotional negative experience blind zone.

5. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The step of generating enhanced interface element prompts for the negative emotional experience blind spot based on its location coordinates in the spatial emotional distribution map includes: The final boundary contour corresponding to each negative emotional experience blind spot is read from the spatial emotion distribution map. The final boundary contour is described by the starting row coordinates, the ending row coordinates, the starting column coordinates, and the ending column coordinates. The starting row coordinates and the ending row coordinates are converted into the starting vertical pixel position and the ending vertical pixel position in the screen pixel coordinate system, and the starting column coordinates and the ending column coordinates are converted into the starting horizontal pixel position and the ending horizontal pixel position in the screen pixel coordinate system to obtain the screen coordinate blind zone rectangle. Based on the starting vertical pixel position, ending vertical pixel position, starting horizontal pixel position, and ending horizontal pixel position of the screen coordinate blind zone rectangle, calculate the center point pixel coordinates of the screen coordinate blind zone rectangle. The vertical coordinate of the center point pixel coordinates is equal to half the sum of the starting vertical pixel position and the ending vertical pixel position, and the horizontal coordinate is equal to half the sum of the starting horizontal pixel position and the ending horizontal pixel position. Obtain the interface layout description file of the current interface of the smart interactive terminal. The interface layout description file contains the element identifiers of all interface elements on the current interface and the element bounding box coordinates of each interface element on the screen. The overlapping area is calculated between the screen coordinate blind zone rectangle and the element bounding box coordinates of each interface element in the interface layout description file. All interface elements that have overlapping areas with the screen coordinate blind zone rectangle are identified, forming a blind zone associated interface element set. For each blind zone associated interface element in the blind zone associated interface element set, the overlapping area between the element bounding box of the blind zone associated interface element and the screen coordinate blind zone rectangle is calculated. Divide the overlapping area by the total area of ​​the bounding boxes of the interface elements associated with the blind zone to obtain the overlapping area ratio of the interface elements associated with the blind zone. The interface elements associated with the blind zone whose overlapping area ratio exceeds a preset overlapping ratio threshold are identified as interface elements mainly affected by the blind zone, and the element identifier of the interface elements mainly affected by the blind zone is recorded. Based on the element identifier of the interface element mainly affected by the blind spot, the set of visual protrusion display parameters corresponding to the corresponding interface element is retrieved from the preset function enhancement effect library. The set of visual protrusion display parameters includes edge highlight color value, edge highlight width value and element scaling ratio value. Based on the element identifiers of the interface elements primarily affected by the blind spot, the corresponding functional guidance prompt text content and functional guidance prompt icon are retrieved from the preset guidance prompt content library. The visual protrusion display parameter set, the functional guidance prompt text content, and the functional guidance prompt icon are encapsulated to generate an element-level enhanced prompt sub-instruction for a single blind spot-related interface element. The above process is repeated to generate a corresponding element-level enhanced prompt sub-instruction for each interface element primarily affected by the blind spot. All element-level enhanced prompt sub-instructions are combined into a complete interface element enhanced prompt instruction. An instruction version number and timestamp information are added to the beginning of the interface element enhanced prompt instruction, and a checksum is added to the end of the interface element enhanced prompt instruction.

6. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The method further includes: After generating the operation intent focus area distribution map, the operation intent focus area distribution map is processed by region segmentation. Regions in the operation intent focus area distribution map whose fusion intent density value is lower than a preset sparse density threshold are identified as density sparse regions. Different region type identifiers are assigned to each density sparse region, and the region type identifiers are attached to the corresponding region pixels in the operation intent focus area distribution map. When drawing voice emotion color markers by superimposing the spectral feature sequence of the voice command, the region type identifier covered by each voice emotion color marker is recorded simultaneously, the total area covered by each voice emotion color marker with the region type identifier being a sparse region is calculated, and the total area is compared with a preset threshold for the area of ​​sparse emotional association regions. If the total area exceeds the threshold of the area of ​​the sparse region of emotional association, the part of the sparse region that overlaps with the negative speech emotion color mark is preferentially marked as a candidate negative emotional experience blind zone. Temporal stability analysis is performed on the candidate negative emotional experience blind zone, and the frequency of occurrence of the candidate negative emotional experience blind zone in the spatial emotion distribution map of multiple consecutive time frames is extracted. If the frequency of occurrence of the candidate negative emotional experience blind zone in multiple consecutive time frames exceeds a preset frequency threshold, then the candidate negative emotional experience blind zone is confirmed as a negative emotional experience blind zone, and the region information of the confirmed negative emotional experience blind zone is fed back to the generation process of the spatial emotional distribution map to adjust the transparency parameter of the voice emotional color marker in the subsequent spatial emotional distribution map. Based on the adjustment results of the transparency parameter, the accuracy of the boundary contour of the subsequently identified negative emotional experience blind zone is dynamically corrected. The corrected boundary contour of the negative emotional experience blind zone is compared with the historical blind zone boundary contour stored in the history record. If the similarity exceeds the preset similarity threshold, the parameter update operation of the blind zone prediction model is triggered.

7. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The method further includes: After generating the spatial emotion distribution map of the fused operation intention and voice emotion, the spatial emotion distribution map is processed by calculating the map entropy value to obtain the overall information entropy value of the spatial emotion distribution map. The difference between the overall information entropy value and the preset benchmark information entropy value is calculated to obtain the information entropy change. The information entropy change is used to characterize the degree of fluctuation of the user's emotional complexity during the interaction process. Based on the positive and negative direction and absolute value of the change in information entropy, it can be determined whether the user's emotional state during the interaction process tends to be more complex or more simple. If the change in information entropy is positive and its absolute value exceeds a preset complexity threshold, it is determined that the user's emotional state is becoming more complex, and a local analysis operation on the spatial emotional distribution map is triggered. In the local analysis operation, the standard deviation of the color channel values ​​of pixels in each predetermined local area of ​​the spatial emotional distribution map is calculated, and the local area with the largest standard deviation value is taken as the key analysis area. Pixel-level analysis is performed on the key analysis area, and the gray value change rate and color channel value change rate of each pixel in the key analysis area are extracted. The correlation between the rate of change of grayscale value and the rate of change of color channel value is calculated to obtain the correlation coefficient between emotion and operation. If the absolute value of the correlation coefficient between emotion and operation is lower than the preset correlation threshold, it is determined that there is a decoupling phenomenon between emotion and operation in the key analysis area. The key analysis area with the decoupling phenomenon between emotion and operation is marked as a potential interactive experience problem area, and the location coordinates of the potential interactive experience problem area are added to the list of areas to be optimized. The list of areas to be optimized is merged with the location coordinates of the negative emotional experience blind spots to generate a comprehensive set of optimized areas, which is used to guide the generation of subsequent interface element enhancement prompts.

8. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The method further includes: After generating the interface element enhancement prompt instruction, while sending the interface element enhancement prompt instruction to the interactive terminal, the sending timestamp and instruction content summary of the interface element enhancement prompt instruction are recorded simultaneously. After the interactive terminal executes the visual protrusion display operation triggered by the interface element enhancement prompt instruction, it collects the user interaction feedback data after execution. The user interaction feedback data includes the update record of the user's touch pressure distribution trajectory on the new interface and the update record of the change in eye gaze heatmap. The user interaction feedback data is processed in real time with sentiment analysis to generate an optimized user sentiment state assessment value. The optimized user sentiment state assessment value is compared with the sentiment state assessment value before the interface element enhancement prompt instruction is sent, and the improvement of sentiment state is calculated. If the improvement in the emotional state exceeds a preset effective improvement threshold, the enhancement prompt strategy corresponding to the interface element enhancement prompt instruction is marked as an effective strategy; if the improvement in the emotional state is lower than the effective improvement threshold, the enhancement prompt strategy corresponding to the interface element enhancement prompt instruction is marked as an invalid strategy. Based on the set of visual salience display parameters used by the enhanced prompt strategy marked as an effective strategy, the weights of the corresponding visual salience display parameter combinations in the preset function enhancement effect library are adjusted, and the weight values ​​of the parameters corresponding to the effective strategies are increased. Based on the feature parameters of the enhancement prompt strategies marked as invalid strategies, the weights of the visual protrusion display parameter set in the preset function enhancement effect library are adjusted to reduce the weight values ​​of the parameters corresponding to invalid strategies. The adjusted weight values ​​are normalized, and the updated set of visual protrusion display parameters and their corresponding weight values ​​are re-stored into the preset function enhancement effect library.

9. The method for optimizing intelligent interactive experience by combining affective computing according to claim 1, characterized in that, The method further includes: During the process of generating the operation intention focus area distribution map, the time series of pressure sensing values ​​of each touch point in the interface touch pressure distribution trajectory is extracted simultaneously, and the time series of pressure sensing values ​​is subjected to frequency domain transformation to obtain the pressure frequency domain feature spectrum. The pressure fluctuation primary frequency component and pressure fluctuation secondary frequency component are identified from the pressure frequency domain feature spectrum, and the frequency value and amplitude of the pressure fluctuation primary frequency component and the pressure fluctuation secondary frequency component are recorded. The frequency value of the pressure fluctuation primary frequency component is matched with a preset tense operation frequency range. If the frequency value of the pressure fluctuation primary frequency component falls into the tense operation frequency range, the user's current operation state is determined to be a tense operation state, and the eye gaze point distribution data of the same period as the tense operation state are extracted from the eye gaze heatmap change record. A gaze point density clustering analysis is performed on the gaze point distribution data during the same period as the tense operation state to generate the coordinates of the gaze point cluster center under the tense state; from the operation intention focusing area distribution map, the region where the fusion intention density value exceeds the preset high density threshold is identified, and the center coordinates of the region are calculated as the high density region center coordinates; the gaze point cluster center coordinates and the high density region center coordinates are offset to obtain the gaze operation offset vector. If the magnitude of the gaze operation offset vector exceeds a preset offset tolerance threshold, it is determined that the user has an abnormal interaction state where the operation and gaze are inconsistent. The time point and location coordinates of the occurrence of the abnormal interaction state are recorded in the abnormal event log. Based on the frequency of the abnormal interaction state recorded in the abnormal event log, the preset density threshold and the preset overlap ratio threshold are dynamically adjusted. The adjusted preset density threshold and the preset overlap ratio threshold are applied to the subsequent process of identifying blind spots in negative emotional experiences and generating interface element enhancement prompts.

10. A smart interactive experience optimization system combining affective computing, characterized in that, The invention includes a processor and a computer-readable storage medium storing machine-executable instructions that, when executed by the processor, implement the intelligent interactive experience optimization method incorporating affective computing as described in any one of claims 1-9.

Images