A method and system for content presentation control of a multi-modal interactive track book

Abstract

The application discloses a kind of multimodal interactive track book content presentation control method and system, it is related to image processing and multimodal man-machine interaction technical field, including acquisition track image sequence and illumination normalization;Extract stable and consistent component in space-time, generate stable structure response and inhibit time-varying interference;Response intensity and main direction are calculated, and area confidence is evaluated;With prior model guide low confidence area peak search and connection, compensate structure and update model;To compensate response does peak screening and connection obtains binary graph, calculates and updates the geometric deviation of model;Deviation is driven when threshold value, spatiotemporal information is fused to generate tracking state vector, and control instruction is analyzed.The application extracts track edge stably under light mutation and shadow interference, and improves interaction continuity by vision-haptics fusion.

Description

Technical Field

[0001] This invention relates to the fields of image processing and multimodal human-computer interaction technology, and in particular to a method and system for controlling the content presentation of a multimodal interactive trackbook. Background Technology

[0002] Track books are interactive readers designed specifically for children. The pages are printed with tracks along which toys or indicators can move. An image recognition module detects the toy's position on the track in real time, driving the content presentation system to play corresponding audio, animation, or light feedback, achieving an immersive "read-as-you-go" experience. With the development of computer vision technology, existing track book products typically use cameras to capture images of the track area and utilize traditional edge detection operators such as Canny and Sobel to extract the track boundaries, thereby locating the toy's real-time position on the track.

[0003] Chinese invention patent application CN119625002A, published on March 14, 2025, discloses an edge detection algorithm for high-speed railway track slabs that is insensitive to changes in light intensity. This algorithm is based on a sliding window detection strategy, determining edge point locations by progressively narrowing the window range. However, this algorithm is primarily designed for high-speed railway track slab processing scenarios, and the computational overhead of the window search strategy is significant, making it difficult to meet the low-latency requirements of real-time interactive track book scenarios. Furthermore, the traditional Canny operator requires manually setting high and low thresholds. When the input image experiences large changes in light intensity or significant fluctuations in the average grayscale value, the pre-set thresholds often fail to adapt to actual working conditions.

[0004] While existing edge detection methods based on phase consistency offer some advantages in terms of insensitivity to changes in lighting and contrast, they are prone to generating false edges and are computationally complex. Therefore, existing track book products experience drastic changes in brightness and contrast in the track area when ambient light changes abruptly (such as curtains opening and closing, lights switching on and off, or cloud movement) or when a child's body / arm casts a moving shadow above the track. This makes it difficult for the image recognition module to effectively extract track boundaries, leading to frequent track tracking loss. Users have described these products as "addictive to certain environments" and "difficult to use." Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a content presentation control method and system for a multimodal interactive track book, aiming to solve the problem of track edge detection failure and interactive interruption caused by sudden changes in ambient light and shadow interference.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for controlling the content presentation of a multimodal interactive trackbook, comprising the following steps: S1. Collect the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence. S2. Jointly extract the temporally stable component and the spatially consistent component from the normalized image sequence. Mark the temporally unstable but spatially consistent component as a time-varying interference mask, and mark the temporally stable and spatially consistent component as a stable structural response basis. Perform local gain suppression on the stable structural response basis based on the time-varying interference mask to obtain the suppressed structural response basis. S3. Calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree. S4. In the low-confidence sub-region where the structural confidence is lower than the preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. The prior model is updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. S5. Perform peak filtering and threshold connection on the compensated structural response basis to obtain the stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated space model. S6. When the geometric deviation exceeds the preset deviation threshold, drive the tactile array to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the analysis content of the track tracking state vector.

[0007] Furthermore, the acquisition of the track area image sequence of the current page of the trackbook, and the inter-frame illumination normalization to obtain a normalized image sequence, includes: The system uses a visual sensor to capture images of the track area on the current page and generates a sequence of track area images. The gray values ​​of each pixel in the track region of the first frame image in the track region image sequence are statistically analyzed. The average gray value of all pixels in the track region of the first frame image is calculated, and the average gray value is marked as the reference gray value. Image frames in the track region image sequence are read frame by frame as image frames to be processed. The gray values ​​of each pixel in the track region of the image frame to be processed are counted. The average gray value of all pixels in the track region of the image frame to be processed is calculated. The average gray value is marked as the gray value of the current frame. The ratio of the reference gray value to the gray value of the current frame is calculated. The ratio is marked as the gray normalization gain coefficient of the current frame. The original grayscale value of each pixel in the image frame to be processed is multiplied with the grayscale normalization gain coefficient of the current frame to obtain the corrected grayscale value of each pixel. The original grayscale value in the image frame to be processed is replaced with the corrected grayscale value of each pixel to obtain the corrected image frame. Using the first image frame as the first frame of the sequence, the corrected image frames are arranged sequentially after the first frame of the sequence according to the original acquisition time sequence corresponding to each corrected image frame, thus generating a normalized image sequence.

[0008] Furthermore, the step of jointly extracting temporally stable and spatially consistent components from the normalized image sequence, marking temporally unstable but spatially consistent components as time-varying interference masks, and marking temporally stable and spatially consistent components as stable structural response bases, and performing local gain suppression on the stable structural response bases based on the time-varying interference masks to obtain suppressed structural response bases, includes: A gray-scale temporal sequence is constructed from the gray values ​​of pixels with the same spatial coordinates in each frame of the normalized image sequence. The standard deviation of the gray values ​​in the gray-scale temporal sequence is used as the temporal fluctuation value. Pixels with temporal fluctuation values ​​less than a preset temporal fluctuation threshold are marked as temporally stable positions, and pixels with temporal fluctuation values ​​greater than or equal to the preset temporal fluctuation threshold are marked as temporally unstable positions. Calculate the phase consistency value pixel by pixel for the last frame of the normalized image sequence to form a spatial phase consistency value map. Mark the pixel positions in the spatial phase consistency value map that have a phase consistency value greater than a preset spatial consistency threshold as spatial consistency positions. The pixel positions that are simultaneously marked as temporally unstable positions and spatially consistent positions are marked as time-varying interference positions, and a time-varying interference mask is constructed with the pixel values ​​of all time-varying interference positions being 1. The pixel positions that are simultaneously marked as stable positions in the temporal domain and consistent positions in the spatial domain are marked as stable structure positions. The phase consistency values ​​at each stable structure position in the spatial phase consistency value map are used as the basis response amplitudes to generate a stable structure response basis. Multiply the amplitude of the base response at the location corresponding to the pixel value of the time-varying interference mask in the stable structural response base by a preset suppression coefficient to obtain the suppressed structural response base.

[0009] Furthermore, the step of calculating the phase consistency value pixel by pixel in the last frame of the normalized image sequence to construct a spatial phase consistency value map includes: Construct a Log-Gabor filter, and convolve the Log-Gabor filter at each scale with the last frame image to obtain the even-symmetric and odd-symmetric filtering responses of each pixel at each scale. Perform arctangent operation on the even-symmetric and odd-symmetric filter responses of each pixel at each scale to obtain the local phase angle of each pixel at each scale; The local amplitude value of each pixel at each scale is obtained by summing the squares of the even-symmetric and odd-symmetric filter responses of each pixel at each scale and taking the square root. The sum of the local amplitude values ​​at all scales for each pixel is obtained. Summing the even-symmetric filter responses of each pixel at each scale yields the sum of even-symmetric responses; summing the odd-symmetric filter responses of each pixel at each scale yields the sum of odd-symmetric responses; performing an arctangent operation on the sum of even-symmetric responses and the sum of odd-symmetric responses yields the combined phase angle. Perform a difference operation on the local phase angle and the overall phase angle of each pixel at each scale to obtain the phase deviation angle; perform a cosine operation on the phase deviation angle to obtain the phase deviation cosine value; perform a multiplication operation on the phase deviation cosine value at each scale and the local amplitude value at the same scale and sum them to obtain the phase consistency numerator; divide the phase consistency numerator by the sum of the local amplitudes to obtain the phase consistency value. A spatial phase consistency value map is constructed using the phase consistency values ​​of all pixel locations.

[0010] Furthermore, the step of calculating the response intensity distribution and principal direction distribution based on the suppressed structural response basis, segmenting the last frame corresponding to the suppressed structural response basis into sub-regions, calculating the matching degree based on the local peak in the response intensity distribution and the principal peak in the principal direction distribution of each sub-region, and generating structural confidence based on the matching degree includes: By performing convolution operations on the suppressed structural response basis using the Sobel operator, the horizontal and vertical gradient values ​​of each pixel are obtained. Perform arctangent operation on the gradient values ​​in the horizontal and vertical directions to obtain the gradient direction angle; The gradient magnitude is obtained by summing the squares of the horizontal and vertical gradient values ​​and then taking the square root. The response intensity distribution map is constructed using the gradient magnitude of all pixels, and the main direction distribution map is constructed using the gradient direction angle of all pixels. Divide the last frame image corresponding to the suppressed structural response basis into sub-regions; determine the angle interval according to the distribution range of all pixel gradient direction angles in the main direction distribution map, and divide the gradient direction interval according to the angle interval; The number of pixels falling into each gradient direction interval is counted in each sub-region, and the angle value corresponding to the gradient direction interval with the most pixels is taken as the main direction and main peak of that sub-region. Within each sub-region, the pixel location in the response intensity distribution map whose gradient magnitude is greater than the gradient magnitude of all pixels within the neighborhood radius is searched and identified as the local peak intensity. The neighborhood radius is determined based on the standard deviation of the gradient magnitude within the sub-region. Calculate the ratio of the number of pixels corresponding to the main peak in the main direction in each sub-region to the total number of pixels in the sub-region, and mark the ratio as the concentration of the main direction in that sub-region. Calculate the ratio of the gradient magnitude of the local intensity peak in each sub-region to the average gradient magnitude of all pixels in the sub-region, and label the ratio as the intensity saliency of that sub-region; multiply the principal direction concentration and the intensity saliency to obtain the matching degree; Sub-regions with a matching degree greater than a preset matching degree threshold are marked as high-confidence sub-regions, and sub-regions with a matching degree less than or equal to the preset matching degree threshold are marked as low-confidence sub-regions, thus forming the structural confidence.

[0011] Furthermore, the preset prior model extracts the node coordinates and adjacency relationships of the track centerline from the track design drawings, calculates and stores the node spacing based on the coordinates of adjacent nodes, and includes track node sequences, node adjacency relationships, and node spacing attributes; in low-confidence sub-regions where the structural confidence is lower than a preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. The prior model is then updated using the constraints of the compensated structural segment to obtain an updated spatial model, generating the compensated structural response basis, including: Obtain the orbit node sequence of the low-confidence sub-region in the prior model, perform linear interpolation on the orbit node sequence, and obtain the local orientation direction of each low-confidence sub-region; Within each low-confidence sub-region, along the local orientation, search for pixel locations on the suppressed structural response substrate where the response intensity is greater than a preset peak threshold to obtain the local peak location set for each low-confidence sub-region. According to the spatial order of the local orientation direction in each low-confidence sub-region, the local peak position sets of each low-confidence sub-region are connected sequentially to form a compensation structure segment; Obtain the node spacing of the prior model, and resample the compensation structure segment at equal intervals according to the node spacing to obtain the coordinates of the resampled nodes; The coordinates of the resampled nodes are compared with the coordinates of the corresponding orbital nodes in the prior model to obtain the position offset of each node. The coordinates of the corresponding nodes in the prior model are then updated with the position offset to obtain the updated spatial model. Extract the node coordinates located in the low-confidence sub-region in the updated spatial model, perform interpolation on each node coordinate to obtain the reconstructed response value of each pixel position in the low-confidence sub-region, and replace the original response value of the corresponding pixel position in the suppressed structural response basis with the reconstructed response value to generate the compensated structural response basis.

[0012] Furthermore, within each low-confidence sub-region, the step of searching for pixel locations with response intensities greater than a preset peak threshold along the local directional direction on the suppressed structural response substrate to obtain a set of local peak locations for each low-confidence sub-region includes: Using the local orientation direction as the search axis direction, the suppressed structural response substrate is sampled at intervals along the search axis direction to obtain the sampling position sequence. The response intensity values ​​of all pixels are counted in the direction perpendicular to the search axis. The sum of the mean and standard deviation of the response intensity values ​​is calculated and used as the threshold for determining local peak values. Pixels whose response intensity values ​​within the window are greater than the local peak determination threshold are marked as candidate peak locations; The candidate peak positions adjacent in the search axis direction are merged into the same peak cluster according to the spatial adjacency relationship. The pixel position corresponding to the maximum value of the response intensity value of all candidate peak positions in the peak cluster is calculated, and the pixel position is taken as the local peak position of the peak cluster. The local peak positions of all peak clusters within each low-confidence sub-region are collected to generate a set of local peak positions.

[0013] Furthermore, the peak filtering and threshold connection are performed on the compensated structural response basis to obtain a stable structural binary map, and the geometric deviation between the stable structural binary map and the updated spatial model is calculated, including: Obtain the response intensity value and gradient direction angle in the principal direction distribution diagram of the compensated structural response base, and perform non-maximum suppression on the compensated structural response base along the gradient direction angle to obtain the non-maximum suppressed response base. By performing double-threshold hysteresis boundary connections on the response basis after nonmaximum suppression, a stable binary structure graph is obtained. Extract the edge point set from the binary graph of the stable structure, extract the node coordinates of the updated spatial model as the model point set, perform iterative nearest point registration on the edge point set and the model point set to obtain the rotation matrix and translation vector; Perform coordinate transformation on the model point set using rotation matrix and translation vector to obtain the transformed model point set. Calculate the mean Euclidean distance between all point pairs of the transformed model point set and the edge point set, and mark the mean Euclidean distance as the geometric deviation.

[0014] Furthermore, when the geometric deviation exceeds a preset deviation threshold, the tactile array is driven to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated from the binary image of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the parsed content of the track tracking state vector, including: The geometric deviation is compared with a preset deviation threshold. If the geometric deviation is greater than the preset deviation threshold, the tactile array is driven to generate a directional vibration sequence, and the generation time of the directional vibration sequence is recorded as a feedback time stamp. The generation time of the binary image of the stable structure is obtained as the generation timescale; Extract all point pairs of Euclidean distances between the edge point set and the transformed model point set, and use all point pairs of Euclidean distances to form the spatial distribution vector of the deviation; The spatial distribution vectors of the feedback timescale, the generation timescale, and the deviation are concatenated to obtain the track tracking status vector. Based on the track tracking state vector, a preset content presentation control command library is retrieved, and the content presentation control command that matches the track tracking state vector is output.

[0015] A content presentation control system for a multimodal interactive trackbook includes: Image acquisition module: used to acquire the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence; Interference suppression module: used to jointly extract temporally stable components and spatially consistent components from normalized image sequences, mark temporally unstable but spatially consistent components as time-varying interference masks, mark temporally stable and spatially consistent components as stable structural response bases, and perform local gain suppression on stable structural response bases based on time-varying interference masks to obtain suppressed structural response bases. Confidence assessment module: used to calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree; Model update module: Used to search for and connect local peaks of the suppressed structural response basis in low-confidence sub-regions where the structural confidence is lower than a preset confidence threshold, guided by the local orientation of the preset prior model, to form a compensated structural segment. The prior model is then updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. Deviation quantization module: used to perform peak filtering and threshold connection on the compensated structural response basis to obtain a stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated spatial model; State analysis module: When the geometric deviation exceeds the preset deviation threshold, it drives the tactile array to generate a directional vibration sequence. It integrates the feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation into a track tracking state vector, and presents control commands based on the analysis content of the track tracking state vector.

[0016] The present invention provides a method and system for controlling the content presentation of a multimodal interactive trackbook, the advantages of which are mainly reflected in the following aspects: 1. This invention effectively suppresses the impact of sudden illumination changes and shadow interference on track edge detection by constructing a dual-domain joint analysis mechanism of "temporally stable + spatially consistent". Specifically, by jointly extracting temporally stable and spatially consistent components from a normalized image sequence, the temporally unstable but spatially consistent components are marked as time-varying interference masks and local gain suppression is implemented. This effectively filters out brightness fluctuations caused by sudden illumination changes and false edges introduced by shadows, while preserving the true edge response of the track structure. Compared to the dependence of the traditional Canny operator on gradient magnitude, this invention uses phase consistency as a measure of spatial consistency, which is insensitive to changes in illumination and contrast, and can maintain a stable edge response even in low-contrast regions.

[0017] 2. This invention also addresses the issue of low confidence in edge detection in shadowed areas through a priori model-guided low-confidence region compensation mechanism. When the confidence of a sub-region structure falls below a threshold, priori topological information (track node sequence, node adjacency, and node spacing) extracted from the track design drawings guides local peak search and connection, forming compensated structural segments and updating the spatial model. This mechanism allows the system to proactively "guess" the track direction in areas with low edge detection confidence, rather than relying entirely on local image information, thus ensuring the continuity of track tracking. Furthermore, this invention drives the haptic array to generate a directional vibration sequence when the geometric deviation exceeds a threshold, fusing the visual tracking state with haptic feedback into a track tracking state vector, achieving compensation for multimodal interaction when the visual modality fails. This solution maintains normal interaction without requiring users to adjust the usage environment and is suitable for various lighting scenarios such as homes, classrooms, and outdoors. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a content presentation control method for a multimodal interactive trackbook according to the present invention. Figure 2 This is a flowchart illustrating the haptic feedback triggering and control command parsing process of the present invention. Figure 3 This is a functional block diagram of a content presentation control system for a multimodal interactive trackbook according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figures 1-2This invention provides a method for controlling the content presentation of a multimodal interactive trackbook, comprising the following steps: S1. Collect the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence. In this embodiment, the process of acquiring the track area image sequence of the current page of the trackbook and performing inter-frame illumination normalization to obtain a normalized image sequence includes: The system uses a visual sensor to capture images of the track area on the current page, generating a sequence of track area images.

[0021] A visual sensor is an image acquisition device installed on a track book reader. Its output is digital image frames. A track area image sequence refers to multiple frames of images continuously acquired by the visual sensor at a fixed frame rate. Each frame contains the track line portion of the current page of the track book. By continuously capturing multiple consecutive track area images and arranging them in the order of acquisition time, a track area image sequence is formed, providing raw image data for subsequent inter-frame illumination normalization.

[0022] The gray values ​​of each pixel in the track region of the first frame image in the track region image sequence are statistically analyzed. The average gray value of all pixels in the track region of the first frame image is calculated, and the average gray value is marked as the baseline gray value.

[0023] The first frame image refers to the earliest captured image in the track area image sequence. The track area refers to the pixel range corresponding to the location of the track line on the trackbook page, pre-defined through calibration. This calibration is performed before the trackbook reader leaves the factory or when the page is first loaded, recording the set of pixel coordinates of the track line in the image coordinate system. Each pixel's grayscale value refers to the brightness value of each pixel in the image; for color images, grayscale conversion is required to extract the brightness component. The average grayscale value of all pixels refers to the arithmetic mean of the grayscale values ​​of all pixels within the track area. The baseline grayscale value serves as a reference standard for grayscale correction of subsequent frames, used to eliminate overall brightness drift between frames caused by changes in ambient lighting.

[0024] Image frames in the track region image sequence are read frame by frame as image frames to be processed. The gray values ​​of each pixel in the track region of the image frame to be processed are counted. The average gray value of all pixels in the track region of the image frame to be processed is calculated. The average gray value is marked as the gray value of the current frame. The ratio of the reference gray value to the gray value of the current frame is calculated. The ratio is marked as the gray normalization gain coefficient of the current frame.

[0025] The image frames to be processed refer to the image frames extracted sequentially from the track region image sequence (excluding the first frame) without undergoing grayscale correction. The grayscale value of the current frame is the arithmetic mean of the grayscale values ​​of all pixels within the track region of the image frame to be processed. The ratio of the reference grayscale value to the grayscale value of the current frame is obtained through division. When the overall brightness of the current frame is lower than that of the first frame, the ratio is greater than 1; otherwise, the ratio is less than 1. This ratio reflects the degree of overall brightness change of the image frame to be processed relative to the first frame within the track region. The grayscale normalization gain coefficient is used for subsequent pixel-by-pixel grayscale adjustment to correct the average brightness of the track region of each frame to the same level as the first frame, thereby eliminating the problem of inconsistent overall brightness between frames caused by changes in ambient lighting.

[0026] The original grayscale value of each pixel in the image frame to be processed is multiplied by the grayscale normalization gain coefficient of the current frame to obtain the corrected grayscale value of each pixel. The original grayscale value in the image frame to be processed is replaced with the corrected grayscale value of each pixel to obtain the corrected image frame.

[0027] The original grayscale value refers to the initial brightness value of each pixel in the image frame to be processed before any grayscale adjustment. The multiplication operation involves multiplying the original grayscale value of each pixel in the image frame to be processed by the same grayscale normalization gain coefficient to obtain the corrected grayscale value for each pixel. The corrected grayscale value makes the average grayscale value of the track region of the entire image frame approximate the reference grayscale value, while the relative grayscale differences between pixels within the frame remain unchanged. After replacing the original grayscale values ​​with the corrected grayscale values ​​of each pixel, a corrected image frame is obtained. The average brightness of this frame in the track region has been normalized to a level matching the first frame.

[0028] Using the first image frame as the first frame of the sequence, the corrected image frames are arranged sequentially after the first frame of the sequence according to the original acquisition time sequence corresponding to each corrected image frame, thus generating a normalized image sequence.

[0029] The first frame of the sequence refers to the first image in the normalized image sequence. The original first frame image is used directly without grayscale correction because its average grayscale value has been set as the baseline grayscale value. The original acquisition sequence refers to the chronological order in which each corrected image frame was initially acquired by the visual sensor. The first frame image and each corrected image frame are arranged sequentially according to their original acquisition sequence to generate a normalized image sequence. In this sequence, the average grayscale value of the track region in all frames is normalized to the same baseline level, eliminating the overall brightness difference between frames caused by sudden changes in ambient light. This provides input data with good illumination consistency for the subsequent extraction of temporally stable components.

[0030] S2. Jointly extract the temporally stable component and the spatially consistent component from the normalized image sequence. Mark the temporally unstable but spatially consistent component as a time-varying interference mask, and mark the temporally stable and spatially consistent component as a stable structural response basis. Perform local gain suppression on the stable structural response basis based on the time-varying interference mask to obtain the suppressed structural response basis. In this embodiment, the step of jointly extracting temporally stable and spatially consistent components from a normalized image sequence, marking temporally unstable but spatially consistent components as time-varying interference masks, and marking temporally stable and spatially consistent components as stable structural response bases, and performing local gain suppression on the stable structural response bases based on the time-varying interference masks to obtain suppressed structural response bases, includes: A gray-scale temporal sequence is constructed from the gray values ​​of pixels with the same spatial coordinates in each frame of the normalized image sequence. The standard deviation of the gray values ​​in the gray-scale temporal sequence is used as the temporal fluctuation value. Pixel positions with temporal fluctuation values ​​less than a preset temporal fluctuation threshold are marked as temporally stable positions, and pixel positions with temporal fluctuation values ​​greater than or equal to the preset temporal fluctuation threshold are marked as temporally unstable positions.

[0031] A grayscale temporal sequence refers to a one-dimensional sequence formed by arranging grayscale values ​​at the same pixel coordinates according to frame number in a normalized image sequence. Temporal fluctuation is measured by the standard deviation of this grayscale temporal sequence; a larger value indicates more drastic fluctuations in grayscale values ​​at that pixel location over time. The preset temporal fluctuation threshold is obtained through statistical analysis of the rate of change in illumination in actual use scenarios of trackbooks. Grayscale temporal samples of trackbook pages are collected under various illumination scenarios, the standard deviation distribution of grayscale values ​​at each pixel location is calculated, and a critical value that can distinguish between static illumination and sudden illumination changes is selected as the threshold. Temporally stable positions refer to pixel locations where the grayscale temporal standard deviation is below the threshold, while temporally unstable positions refer to pixel locations where the grayscale temporal standard deviation is above or equal to the threshold.

[0032] Calculate the phase consistency value pixel by pixel for the last frame of the normalized image sequence to form a spatial phase consistency value map. Mark the pixel positions in the spatial phase consistency value map that have a phase consistency value greater than a preset spatial consistency threshold as spatial consistency positions.

[0033] Phase coherence value is a numerical value in the frequency domain that measures the degree of phase consistency among different frequency components of an image at a given pixel location. Its value ranges from 0 to 1, with values ​​closer to 1 indicating the presence of stable structural edges at that location. The phase coherence value is calculated based on the following formula: In the formula, pixel position Phase consistency value at the location; This is the filter scale index, ranging from 1 to... , The total number of scales; This represents the local amplitude value at the nth scale position for that pixel. For the first The local phase angle at this pixel location at this scale; This is the weighted average of the local phase angles at each scale; To prevent the addition of extremely small positive numbers when the denominator is zero, a value of 10 is usually used. 6 The last frame of a normalized image sequence refers to the last frame of the image sequence.

[0034] The spatial phase consistency value map refers to a two-dimensional matrix composed of the phase consistency values ​​of all pixels in the last frame arranged by coordinates. The preset spatial consistency threshold is determined based on the statistical distribution of phase consistency of the track book page samples. Specifically, it is determined by statistically analyzing the phase consistency values ​​of track edge positions and non-track area positions, and selecting the value that minimizes the classification error between the two types of regions as the threshold. Spatial consistency position refers to the pixel position in the last frame where the phase consistency value is higher than the threshold.

[0035] Pixel positions that are simultaneously marked as temporally unstable positions and spatially consistent positions are marked as time-varying interference positions, and a time-varying interference mask is constructed by setting the pixel value of all time-varying interference positions to 1.

[0036] Time-varying interference locations refer to pixel locations that simultaneously satisfy both temporal instability and spatial consistency. Physically, they are regions affected by sudden changes in illumination or moving shadows, yet simultaneously exhibiting edge structure characteristics; in other words, interference regions where shadow boundaries are mistakenly identified as track edges. A time-varying interference mask is a binary matrix with the same size as the last frame of the normalized image sequence, where values ​​are 1 at time-varying interference locations and 0 at all other locations.

[0037] The pixel positions that are simultaneously marked as stable positions in the temporal domain and consistent positions in the spatial domain are marked as stable structure positions. The phase consistency values ​​at each stable structure position in the spatial phase consistency value map are used as the basis response amplitudes to generate a stable structure response basis.

[0038] A stable structural location refers to a pixel location that simultaneously satisfies both temporal stability and spatial consistency. Physically, it refers to a region with minimal grayscale fluctuations over time and phase consistency in space. The basis response amplitude is taken as the phase consistency value of this stable structural location in the spatial phase consistency value map. The stable structural response basis is a two-dimensional matrix formed by arranging the basis response amplitudes of all stable structural locations according to their coordinates; the response amplitudes of unstable structural locations are set to zero.

[0039] Multiply the amplitude of the base response at the location corresponding to the pixel value of the time-varying interference mask in the stable structural response base by a preset suppression coefficient to obtain the suppressed structural response base.

[0040] The preset suppression coefficient refers to the multiplication factor obtained through experimental calibration, used to reduce the response amplitude at time-varying interference locations, and its value ranges from 0 to 1. After suppression, when the time-varying interference mask is set to 1 at a certain pixel location, the base response amplitude is multiplied by the preset suppression coefficient; when the time-varying interference mask is set to 0, the original base response amplitude at that location remains unchanged. After local gain suppression, the false edge response introduced by time-varying interference is reduced, while the response of the real orbital structure is preserved.

[0041] In this embodiment, the step of calculating the phase consistency value pixel by pixel in the last frame of the normalized image sequence to construct a spatial phase consistency value map includes: A Log-Gabor filter is constructed, and the Log-Gabor filter at each scale is convolved with the last frame image to obtain the even-symmetric and odd-symmetric filtering responses of each pixel at each scale.

[0042] Specifically, the Log-Gabor filter is a bandpass filter that exhibits a Gaussian distribution on the logarithmic frequency scale in the frequency domain. Its transfer function has a symmetrical form on the logarithmic frequency axis, effectively extracting local frequency information at different scales in an image. During the construction process, the scale parameters are uniformly distributed at logarithmic intervals, with 4 to 6 typical scales. The orientation parameters are distributed at equal intervals from 0 to π, with 6 to 8 typical orientations, to cover different frequency bands and orientations.

[0043] Convolution is performed in the frequency domain through Fourier transform. Specifically, Fourier transform is performed on the last frame image, and after dot product with the frequency domain transfer function of the filter, it is returned to the spatial domain by inverse Fourier transform. The real part of the convolution result is an even-symmetric filter response, and the imaginary part is an odd-symmetric filter response.

[0044] Perform arctangent operations on the even-symmetric and odd-symmetric filter responses of each pixel at each scale to obtain the local phase angle of each pixel at each scale.

[0045] Specifically, the arctangent operation refers to performing a four-quadrant arctangent function operation on the even-symmetric and odd-symmetric filter responses at the same pixel location. The quadrant to which the angle belongs is determined by the ratio of the odd-symmetric response to the even-symmetric response and its sign. The local phase angle refers to the angle value obtained after the arctangent operation, which ranges from -π to π, representing the phase offset of the local structure of the image at the nth scale relative to the filter.

[0046] The local amplitude value of each pixel at each scale is obtained by summing the squares of the even-symmetric and odd-symmetric filter responses at each scale and then taking the square root.

[0047] Specifically, the local amplitude value refers to the square root of the sum of the squares of the even-symmetric and odd-symmetric filter responses at the same pixel location, reflecting the energy intensity of the local frequency component at the nth scale. The larger this value, the more significant the contribution of that frequency component to the local structure of the image.

[0048] The sum of the local amplitude values ​​at all scales for each pixel is obtained.

[0049] Specifically, the local amplitude summation refers to the cumulative value obtained by adding the local amplitude values ​​at all scales at the same pixel location. This summation is used as the normalized denominator for subsequent phase coherence value calculation. This summation uniformly quantizes the energy of each frequency component, so that the final phase coherence value is constrained to a value range of 0 to 1.

[0050] Summing the even-symmetric filter responses of each pixel at each scale yields the sum of even-symmetric responses; summing the odd-symmetric filter responses of each pixel at each scale yields the sum of odd-symmetric responses; performing an arctangent operation on the sum of even-symmetric responses and the sum of odd-symmetric responses yields the combined phase angle.

[0051] Specifically, the sum of even-symmetric responses refers to the value obtained by adding the even-symmetric filtered responses at all scales for the same pixel location; the sum of odd-symmetric responses refers to the value obtained by adding the odd-symmetric filtered responses at all scales for the same pixel location. The synthesized phase angle is the angle value obtained after performing a four-quadrant arctangent operation on the sum of even-symmetric and odd-symmetric responses, reflecting the average phase direction after synthesizing across all scales. This synthesized phase angle serves as a reference benchmark for measuring the consistency of local phase angles across different scales.

[0052] Perform a difference operation on the local phase angle and the overall phase angle of each pixel at each scale to obtain the phase deviation angle; perform a cosine operation on the phase deviation angle to obtain the phase deviation cosine value; perform a multiplication operation on the phase deviation cosine value at each scale and the local amplitude value at the same scale and sum them to obtain the phase consistency numerator; divide the phase consistency numerator by the sum of the local amplitudes to obtain the phase consistency value.

[0053] Specifically, the phase deviation angle refers to the difference between the local phase angle and the composite phase angle, which quantifies the degree of phase deviation of each frequency component at that location. The phase deviation cosine value refers to the value obtained by performing a cosine operation on the phase deviation angle, which, combined with the absolute value of the sine of the phase deviation angle, constitutes a measure of phase consistency. The phase consistency numerator refers to the sum of the product of the local amplitude values ​​at each scale and the phase deviation cosine value. Dividing this numerator by the sum of the local amplitudes yields the phase consistency value at that pixel location. The closer this value is to 1, the more consistent the phase of each frequency component.

[0054] A spatial phase consistency value map is constructed using the phase consistency values ​​of all pixel locations.

[0055] Specifically, the spatial phase consistency value map refers to a two-dimensional data matrix composed of the phase consistency values ​​calculated from all pixel positions in the last frame image, arranged according to the original pixel coordinates. This matrix has the same number of rows and columns as the last frame image, and the value of each element in the matrix corresponds to the salience of the structural edge at that pixel position. The higher the value, the greater the probability that a stable structural edge exists at that position.

[0056] S3. Calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree. In this embodiment, the step of calculating the response intensity distribution and principal direction distribution based on the suppressed structural response basis, segmenting the last frame corresponding to the suppressed structural response basis into sub-regions, calculating the matching degree based on the local peak in the response intensity distribution and the principal peak in the principal direction distribution of each sub-region, and generating structural confidence based on the matching degree includes: The horizontal and vertical gradient values ​​of each pixel are obtained by convolving the suppressed structural response basis with the Sobel operator.

[0057] Specifically, the Sobel operator is a discrete differential operator consisting of two sets of 3x3 integer coefficient matrices. The horizontal template is [-1,0,1; -2,0,2; -1,0,1], and the vertical template is [-1,-2,-1; 0,0,0; 1,2,1], used to calculate the approximate first derivative values ​​of the image in the horizontal and vertical directions, respectively. The convolution operation involves performing a weighted summation operation between the horizontal and vertical templates of the Sobel operator and the values ​​of each pixel position and its 3x3 neighboring pixels in the suppressed structural response basis.

[0058] The horizontal gradient value refers to the value output after horizontal Sobel template convolution, representing the rate of change of the response intensity at that pixel location along the horizontal direction of the image; the vertical gradient value refers to the value output after vertical Sobel template convolution, representing the rate of change of the response intensity at that pixel location along the vertical direction of the image.

[0059] Perform arctangent operations on the horizontal and vertical gradient values ​​to obtain the gradient direction angle.

[0060] Specifically, the arctangent operation refers to performing a four-quadrant arctangent function atan2 on the vertical and horizontal gradient values ​​at the same pixel location, and determining the quadrant of the angle based on the ratio and sign of the two values. The gradient direction angle refers to the angle value obtained after the arctangent operation, and its value ranges from -π / 2 to π / 2. It represents the direction of the largest change rate of response intensity at the pixel location, and this direction is perpendicular to the orientation of the track edge.

[0061] The gradient magnitude is obtained by summing the squares of the horizontal and vertical gradient values ​​and then taking the square root.

[0062] Specifically, gradient magnitude refers to the value obtained by squaring the horizontal and vertical gradient values ​​at the same pixel location, summing them, and then taking the square root. Its calculation reflects the degree of change in response intensity at that pixel location. The larger the value, the more prominent the structural edge at that location.

[0063] The response intensity distribution map is constructed using the gradient magnitudes of all pixels, and the principal direction distribution map is constructed using the gradient direction angles of all pixels.

[0064] Specifically, the response intensity distribution map is a two-dimensional data matrix formed by arranging the gradient magnitudes of all pixel locations according to their pixel coordinates. This matrix represents the intensity distribution of structural edges at various points in the suppressed structural response substrate. The principal orientation distribution map is a two-dimensional data matrix formed by arranging the gradient orientation angles of all pixel locations according to their pixel coordinates. This matrix represents the orientation distribution of structural edges at various points in the suppressed structural response substrate.

[0065] Divide the last frame image corresponding to the suppressed structural response basis into sub-regions; determine the angle interval according to the distribution range of all pixel gradient direction angles in the main direction distribution map, and divide the gradient direction interval according to the angle interval.

[0066] Specifically, a sub-region refers to a non-overlapping rectangular image block formed by dividing the last frame image into a predetermined number of rows and columns. A typical segmentation method is to divide the image into 8x8 or 16x16 sub-regions, each of the same size and covering all pixel positions of the last frame image. The angular interval refers to the width of the interval calculated based on the numerical distribution range of the gradient direction angles of all pixels in the main direction distribution map. It is calculated by dividing the gradient direction angle range from -π / 2 to π / 2 into a predetermined number of equal parts, typically 36 or 72. The width of each of these equal parts is the angular interval. A gradient direction interval refers to multiple consecutive angular ranges divided by the angular interval, used to statistically analyze the distribution of pixel gradient direction angles within each sub-region.

[0067] The number of pixels falling into each gradient direction interval is counted in each sub-region, and the angle value corresponding to the gradient direction interval with the most pixels is taken as the main direction and main peak of that sub-region.

[0068] Specifically, the number of pixels refers to the total number of pixels in each sub-region whose gradient direction angle values ​​fall within a certain gradient direction interval. The main direction peak refers to the angle value corresponding to the interval with the most pixels in each gradient direction interval. This angle value represents the edge orientation of most pixels in the sub-region, that is, the main structural direction of the sub-region.

[0069] Within each sub-region, the pixel location in the response intensity distribution map whose gradient magnitude is greater than the gradient magnitude of all pixels within the neighborhood radius is searched and identified as the local peak intensity. The neighborhood radius is determined based on the standard deviation of the gradient magnitude within the sub-region.

[0070] Specifically, the neighborhood radius refers to half the side length of the square neighborhood centered on the candidate pixel location. This radius value is determined by a linear mapping of the standard deviation of the gradient magnitudes of all pixels within the sub-region. The local intensity peak refers to the pixel location within the neighborhood radius whose gradient magnitude is greater than the gradient magnitudes of all neighboring pixels; this location is the point with the strongest response intensity in that local region.

[0071] Calculate the ratio of the number of pixels corresponding to the main peak in the main direction in each sub-region to the total number of pixels in the sub-region, and mark the ratio as the concentration of the main direction in that sub-region.

[0072] Specifically, the concentration of the main direction refers to the ratio of the number of pixels corresponding to the main peak of the main direction to the total number of pixels in the sub-region. The larger the ratio, the more concentrated the gradient direction of the pixels in the sub-region is near the main peak of the main direction, and the more consistent the structure orientation.

[0073] Calculate the ratio of the gradient magnitude of the local intensity peak in each sub-region to the average gradient magnitude of all pixels in the sub-region, and mark the ratio as the intensity saliency of the sub-region; multiply the principal direction concentration and the intensity saliency to obtain the matching degree.

[0074] Specifically, the gradient magnitude of a local intensity peak refers to the gradient magnitude at the location of the local intensity peak within that sub-region, while the average gradient magnitude of all pixels within that sub-region refers to the arithmetic mean of the gradient magnitudes of all pixels within that sub-region. Intensity saliency is the ratio of the gradient magnitude of the local intensity peak to the average gradient magnitude of all pixels; a larger ratio indicates that the local structure intensity within that sub-region is more prominent relative to the overall background. Matching degree is the product of the principal direction concentration and the intensity saliency, calculated based on the following formula: In the formula, The matching degree of the sub-region; Concentration in the main direction; The intensity significance is the key factor. This matching degree comprehensively reflects the structural quality of the sub-region in terms of both directional consistency and intensity significance. A higher matching degree indicates a more reliable structural response in the region.

[0075] Sub-regions with a matching degree greater than a preset matching degree threshold are marked as high-confidence sub-regions, and sub-regions with a matching degree less than or equal to the preset matching degree threshold are marked as low-confidence sub-regions, thus forming the structural confidence.

[0076] Specifically, the preset matching degree threshold is determined by statistical analysis of the matching degree values ​​of track book page samples. Multiple track book page images under different lighting conditions are collected, the matching degree of each sub-region is calculated and its edge detection results are marked as correct, the receiver operating characteristic curve is plotted, and the value that results in the highest correct classification rate is selected as the threshold.

[0077] High-confidence sub-regions refer to sub-regions with a matching degree greater than the threshold, indicating that the orbital structure in this region is clear, oriented consistently, and has significant intensity, and its structural response can be directly relied upon. Low-confidence sub-regions refer to sub-regions with a matching degree less than or equal to the threshold, indicating that this region is affected by shadow coverage or insufficient contrast, and its structural response has lower confidence. The confidence labels of all sub-regions together constitute the structural confidence.

[0078] S4. In the low-confidence sub-region where the structural confidence is lower than the preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. The prior model is updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. In this embodiment, the preset prior model extracts the node coordinates and adjacency relationships of the track centerline from the track design drawings, calculates and stores the node spacing based on the adjacent node coordinates, and includes track node sequences, node adjacency relationships, and node spacing attributes. Within low-confidence sub-regions where the structural confidence level is below a preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. This compensated structural segment constraint updates the prior model, resulting in an updated spatial model and generating the compensated structural response basis, including: Obtain the orbit node sequence of the low-confidence sub-region in the prior model, and perform linear interpolation on the orbit node sequence to obtain the local orientation direction of each low-confidence sub-region.

[0079] Specifically, the prior model refers to the topology of the track centerline extracted from the digital file of the track design drawings. It is constructed by sampling along the path direction at equal arc length intervals to obtain a sequence of track nodes, recording the image coordinates of each node, the connection relationships between adjacent nodes, and their Euclidean distances. The low-confidence sub-region track node sequence refers to the continuous sub-sequence formed by selecting nodes whose coordinates fall within the boundary of the low-confidence sub-region from the node sequence of the prior model, along with their adjacent nodes.

[0080] Linear interpolation refers to inserting intermediate points between adjacent nodes at equal intervals of one pixel, so that the interpolated nodes form a continuous polyline in the image space. The local direction refers to the angle of the tangent direction of the polyline segment at each interpolation point, which is obtained by calculating the arctangent of the coordinate difference between adjacent interpolation points.

[0081] Within each low-confidence sub-region, pixel locations with response intensities greater than a preset peak threshold are searched along the local orientation on the suppressed structural response substrate to obtain the local peak location set for each low-confidence sub-region.

[0082] Specifically, the preset peak threshold refers to the sum of the mean and standard deviation of the response intensity of all pixels in the suppressed structural response base. This sum is used as a judgment threshold, ensuring that pixels with response intensities significantly higher than the average level are retained. The local peak location set refers to a one-dimensional search along the local direction within each low-confidence sub-region. Pixel locations with response intensities exceeding the threshold along the search path are clustered according to spatial adjacency. The pixel location with the highest response intensity in each cluster is taken as the representative point, and all representative points constitute this set.

[0083] According to the spatial order of the local orientation direction in each low-confidence sub-region, the local peak position sets of each low-confidence sub-region are connected sequentially to form a compensation structure segment.

[0084] Specifically, spatial order refers to the arrangement order obtained by sorting the local peak positions according to the local orientation of the track, and compensation structure segment refers to the continuous broken line obtained by connecting adjacent local peak positions in sequence with straight line segments according to the spatial order. This broken line represents the track structure outline reconstructed by the prior model in the low confidence sub-region.

[0085] Obtain the node spacing of the prior model, and resample the compensation structure segment at equal intervals according to the node spacing to obtain the coordinates of the resampled nodes.

[0086] Specifically, node spacing refers to the Euclidean distance between adjacent track nodes that has been calculated and stored during the prior model construction. Equal-distance resampling refers to sequentially extracting sampling points along a polyline path starting from the beginning of the compensation structure segment, using this node spacing as a fixed arc length step, so that the length of the polyline path between any two adjacent sampling points is equal to the node spacing. The resampling node coordinates refer to the horizontal and vertical coordinates of each sampling point in the image coordinate system after equal-distance resampling.

[0087] The coordinates of the resampled nodes are compared with the coordinates of the corresponding orbital nodes in the prior model to obtain the position offset of each node. The coordinates of the corresponding nodes in the prior model are then updated with the position offsets to obtain the updated spatial model.

[0088] Specifically, the coordinates of the corresponding orbital node refer to the original orbital node positions in the prior model that have the same sequence number as the resampled node. The difference operation involves calculating the differences between the horizontal and vertical coordinates of the resampled node and the original node, respectively, to obtain the horizontal and vertical offsets. The position offsets reflect the deviation of the actual imaging position of the orbital node from the predicted position of the prior model. The original coordinates of all nodes in the prior model are added to the corresponding position offsets to obtain the updated coordinates of each node. All updated nodes constitute the updated spatial model.

[0089] Extract the node coordinates located in the low-confidence sub-region in the updated spatial model, perform interpolation on each node coordinate to obtain the reconstructed response value of each pixel position in the low-confidence sub-region, and replace the original response value of the corresponding pixel position in the suppressed structural response basis with the reconstructed response value to generate the compensated structural response basis.

[0090] Specifically, the node coordinates within the low-confidence sub-region refer to the coordinates of all orbital nodes whose spatial locations fall within the boundary of the low-confidence sub-region in the updated spatial model. The interpolation operation involves performing bilinear interpolation on the node coordinates and their corresponding response intensity values ​​within the low-confidence sub-region. Using the node coordinates as control points, the estimated response intensity is calculated based on the weighted average distance between each pixel location and its surrounding nodes. The reconstructed response value refers to the response intensity value obtained after bilinear interpolation at each pixel location. Replacing the original response values ​​of all pixels within the low-confidence sub-region with this value generates the compensated structural response basis.

[0091] In this embodiment, the step of searching for pixel locations with response intensities greater than a preset peak threshold on the suppressed structural response substrate along the local directional direction within each low-confidence sub-region to obtain the local peak location set of each low-confidence sub-region includes: Using the local orientation direction as the search axis direction, the suppressed structural response substrate is sampled at intervals along the search axis direction to obtain the sampling position sequence.

[0092] Specifically, the search axis direction refers to a virtual straight line drawn on the image plane with the angle of the local direction of movement as the orientation, pointing towards the geometric center of the low-confidence sub-region. Interval sampling refers to extracting pixel positions point by point along this virtual straight line with a step size of one pixel, so that the sampling points continuously cover the entire low-confidence sub-region along the axis. The sampling position sequence refers to the set of position coordinates of all sampling points extracted along the search axis direction, arranged in order from the start point to the end point.

[0093] The response intensity values ​​of all pixels are counted in the direction perpendicular to the search axis. The sum of the mean and standard deviation of the response intensity values ​​is calculated and used as the threshold for determining local peak values.

[0094] Specifically, the direction perpendicular to the search axis refers to the direction within the image plane that forms a 90-degree angle with the search axis. The response intensity value of all pixels refers to the response intensity values ​​of all pixels in the strip region formed by extending three pixels on each side of each sampling point along the search axis in the low-confidence sub-region. The sum of the mean and standard deviation refers to the value obtained by adding the arithmetic mean and standard deviation of all response intensity values ​​in this strip region. This sum is used as the local peak determination threshold.

[0095] Pixels whose response intensity values ​​within the window are greater than the local peak determination threshold are marked as candidate peak locations.

[0096] Specifically, a window refers to a rectangular neighborhood centered on each sampling position along the search axis, with a width of one pixel along the axial direction and a height of seven pixels along the vertical direction. Candidate peak positions refer to the pixel positions within the window where the response intensity value is greater than the local peak determination threshold. These positions represent candidate points where the response intensity locally spikes.

[0097] Adjacent candidate peak positions along the search axis are grouped into the same peak cluster according to their spatial adjacency. The pixel position corresponding to the maximum response intensity value of all candidate peak positions within the peak cluster is calculated, and this pixel position is taken as the local peak position of the peak cluster.

[0098] Specifically, spatial adjacency refers to the fact that two candidate peak positions are considered adjacent when the pixel distance between them along the search axis is less than a preset connectivity distance, which is preset to two pixel units and determined by the minimum linewidth of the track. Peak cluster refers to the grouping formed by merging all candidate peak positions that are adjacent to each other along the search axis into the same set. Local peak position refers to the candidate peak position with the largest response intensity value found within the peak cluster, and this position is used as the representative point of the cluster.

[0099] The local peak positions of all peak clusters within each low-confidence sub-region are collected to generate a set of local peak positions.

[0100] Specifically, aggregation refers to merging the local peak positions obtained from each peak cluster in all low-confidence sub-regions into the same set. The local peak position set refers to the complete set consisting of all local peak positions in all low-confidence sub-regions. Each element in this set represents a reliable candidate point in the suppressed structural response base where the response intensity is significant and distributed along the trajectory.

[0101] S5. Perform peak filtering and threshold connection on the compensated structural response basis to obtain the stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated space model. In this embodiment, the step of performing peak filtering and threshold connection on the compensated structural response basis to obtain a stable structural binary map, and calculating the geometric deviation between the stable structural binary map and the updated spatial model, includes: Obtain the response intensity value and gradient direction angle in the principal direction distribution diagram of the compensated structural response basis, and perform non-maximum suppression on the compensated structural response basis along the gradient direction angle to obtain the non-maximum suppressed response basis.

[0102] Specifically, the response intensity value refers to the numerical value of each pixel position in the compensated structural response base. The gradient direction angle refers to the angle value at the corresponding pixel position in the principal direction distribution map. Non-maximum suppression refers to performing a neighborhood comparison on each pixel in the compensated structural response base along the gradient direction angle direction, retaining the pixel with the largest response intensity in the neighborhood along the gradient direction angle direction, and setting the response intensity of the remaining pixels to zero. This operation refines the structural edges to a single pixel width.

[0103] By performing double-threshold hysteresis boundary connections on the response basis after nonmaximum suppression, a stable binary structure graph is obtained.

[0104] Specifically, dual-threshold hysteresis boundary connection refers to setting two judgment thresholds, a high threshold and a low threshold, on the response basis after non-maximum suppression. The high threshold is obtained by calculating the sum of the mean and twice the standard deviation of the response intensity values ​​of all non-zero pixels in the response basis after non-maximum suppression, and the low threshold is half of the high threshold.

[0105] Pixels with response intensities higher than the high threshold are marked as strong edge pixels and retained. Pixels with response intensities between the low and high thresholds are marked as weak edge pixels. Weak edge pixels are retained only when they are connected to the eight-neighborhood of strong edge pixels, and the rest are set to zero. Finally, a binary image is obtained with the retained pixel positions set to 1 and the rest pixel positions set to 0.

[0106] Extract the edge point set from the binary graph of the stable structure, extract the node coordinates of the updated spatial model as the model point set, and perform iterative nearest point registration between the edge point set and the model point set to obtain the rotation matrix and translation vector.

[0107] Specifically, the edge point set refers to the set of coordinates of all pixels with a value of 1 in the stable structure binary image. The model point set refers to the set of coordinates of all orbital nodes in the update space model. Iterative nearest point registration refers to an iterative optimization algorithm that performs rigid transformation alignment on the two point sets. In each iteration, for each point in the edge point set, the corresponding point in the model point set with the closest Euclidean distance is found to establish a matching relationship. The rotation matrix and translation vector are solved by minimizing the sum of squared Euclidean distances of all corresponding point pairs. After iteration until convergence, the final rotation matrix and translation vector are output. The rotation matrix is ​​an orthogonal matrix in a two-dimensional plane, used to describe the angle of rotation of the point set around the origin; the translation vector is a two-dimensional column vector used to describe the translation amount of the point set in the horizontal and vertical directions. Its registration objective function is as follows: In the formula, Let be the rotation matrix to be solved, which is a 2x2 orthogonal matrix; Let be the translation vector to be solved, which is a 2x1 column vector; This represents the total number of corresponding point pairs between the edge point set and the model point set. For edge point set The first in Coordinates of a point; For model point set Zhongyu The corresponding nearest neighbor coordinates; This is the Euclidean distance operator, representing the straight-line distance between two points.

[0108] Perform coordinate transformation on the model point set using rotation matrix and translation vector to obtain the transformed model point set. Calculate the mean Euclidean distance between all point pairs of the transformed model point set and the edge point set, and mark the mean Euclidean distance as the geometric deviation.

[0109] Specifically, coordinate transformation refers to the transformation of the coordinates of each node in the model point set. Perform operation ,in For the transformed number The coordinates of each node. The transformed model point set refers to the coordinates of all orbital nodes after coordinate transformation. The set of points, the mean Euclidean distance of all point pairs refers to the arithmetic mean of the Euclidean distances between the corresponding point pairs of the transformed model point set and the edge point set when the iterative nearest point registration finally converges. The geometric deviation refers to the mean Euclidean distance. The smaller the value, the better the edge in the stable structure binary map matches the orbital structure of the updated space model.

[0110] S6. When the geometric deviation exceeds the preset deviation threshold, drive the tactile array to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the analysis content of the track tracking state vector.

[0111] In this embodiment, when the geometric deviation exceeds a preset deviation threshold, the tactile array is driven to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated from the binary image of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the parsed content of the track tracking state vector, including: The geometric deviation is compared with a preset deviation threshold. If the geometric deviation is greater than the preset deviation threshold, the tactile array is driven to generate a directional vibration sequence, and the generation time of the directional vibration sequence is recorded as a feedback time stamp.

[0112] Specifically, the preset deviation threshold refers to the upper limit of geometric deviation determined through calibration experiments of the track book product. The calibration method involves collecting multiple sets of geometric deviation samples under normal interactive conditions of the track book, and calculating the sum of the sample mean and three times the standard deviation as the threshold. The tactile array refers to a hardware device composed of multiple vibration units arranged in rows and columns. Each vibration unit is independently controlled. The directional vibration sequence refers to the time-series signal generated by the vibration units triggered sequentially along the track offset direction. The feedback time stamp refers to the moment when the tactile array begins outputting the directional vibration sequence, recorded with millisecond-level precision.

[0113] The generation time of the stable structure binary graph is obtained as the generation timescale.

[0114] Specifically, the generation time stamp refers to the moment when the stable structure binary image is output after completing the double-threshold hysteresis boundary connection in the image processing pipeline. This moment is calculated by adding the frame timestamp of image acquisition to the image processing time.

[0115] Extract all point pairs of Euclidean distances between the edge point set and the transformed model point set, and use all point pairs of Euclidean distances to form the spatial distribution vector of the deviation.

[0116] Specifically, the Euclidean distance between all point pairs in the edge point set and the transformed model point set refers to the Euclidean distance between the coordinates of each node in the transformed model point set and its nearest neighbor in the edge point set when the iterative nearest point registration finally converges. The spatial distribution vector refers to a one-dimensional vector formed by arranging the Euclidean distance values ​​of all point pairs in the spatial arrangement order of the corresponding nodes in the model point set. This vector retains the magnitude and distribution information of the deviation at each position of the track.

[0117] The spatial distribution vectors of the feedback timescale, the generation timescale, and the deviation are concatenated to obtain the track tracking state vector.

[0118] Specifically, vector concatenation refers to adding the feedback timescale and the generation timescale as scalar elements to the head of the spatial distribution vector of the deviation, forming a one-dimensional vector with an added dimension. The track tracking state vector is the composite vector obtained after vector concatenation, where the first element is the feedback timescale, the second element is the generation timescale, and the remaining elements are the components of the spatial distribution vector of the deviation.

[0119] Based on the track tracking state vector, a preset content presentation control command library is retrieved, and the content presentation control command that matches the track tracking state vector is output.

[0120] Specifically, the preset content presentation control command library refers to a pre-established mapping table that corresponds the deviation distribution pattern in the track tracking state vector to the content presentation operation commands. It is constructed by configuring corresponding audio playback, animation display, or light feedback commands for the spatial distribution vector of deviations with different amplitudes and distribution characteristics. The retrieval process calculates the Euclidean distance between the spatial distribution vector of the deviation in the track tracking state vector and each entry in the command library, selecting the content presentation control command corresponding to the entry with the smallest distance as the output.

[0121] Please see Figure 3 This invention provides a content presentation control system for a multimodal interactive trackbook, comprising: Image acquisition module: used to acquire the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence; Interference suppression module: used to jointly extract temporally stable components and spatially consistent components from normalized image sequences, mark temporally unstable but spatially consistent components as time-varying interference masks, mark temporally stable and spatially consistent components as stable structural response bases, and perform local gain suppression on stable structural response bases based on time-varying interference masks to obtain suppressed structural response bases. Confidence assessment module: used to calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree; Model update module: Used to search for and connect local peaks of the suppressed structural response basis in low-confidence sub-regions where the structural confidence is lower than a preset confidence threshold, guided by the local orientation of the preset prior model, to form a compensated structural segment. The prior model is then updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. Deviation quantization module: used to perform peak filtering and threshold connection on the compensated structural response basis to obtain a stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated spatial model; State analysis module: When the geometric deviation exceeds the preset deviation threshold, it drives the tactile array to generate a directional vibration sequence. It integrates the feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation into a track tracking state vector, and presents control commands based on the analysis content of the track tracking state vector.

[0122] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for controlling the content presentation of a multimodal interactive trackbook, characterized in that, Includes the following steps: S1. Collect the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence. S2. Jointly extract the temporally stable component and the spatially consistent component from the normalized image sequence. Mark the temporally unstable but spatially consistent component as a time-varying interference mask, and mark the temporally stable and spatially consistent component as a stable structural response basis. Perform local gain suppression on the stable structural response basis based on the time-varying interference mask to obtain the suppressed structural response basis. S3. Calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree. S4. In the low-confidence sub-region where the structural confidence is lower than the preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. The prior model is updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. S5. Perform peak filtering and threshold connection on the compensated structural response basis to obtain the stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated space model. S6. When the geometric deviation exceeds the preset deviation threshold, drive the tactile array to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the analysis content of the track tracking state vector.

2. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The acquisition of the track area image sequence on the current page of the trackbook, followed by inter-frame illumination normalization, yields a normalized image sequence, including: The system uses a visual sensor to capture images of the track area on the current page and generates a sequence of track area images. The gray values ​​of each pixel in the track region of the first frame image in the track region image sequence are statistically analyzed. The average gray value of all pixels in the track region of the first frame image is calculated, and the average gray value is marked as the reference gray value. Image frames in the track region image sequence are read frame by frame as image frames to be processed. The gray values ​​of each pixel in the track region of the image frame to be processed are counted. The average gray value of all pixels in the track region of the image frame to be processed is calculated. The average gray value is marked as the gray value of the current frame. The ratio of the reference gray value to the gray value of the current frame is calculated. The ratio is marked as the gray normalization gain coefficient of the current frame. The original grayscale value of each pixel in the image frame to be processed is multiplied with the grayscale normalization gain coefficient of the current frame to obtain the corrected grayscale value of each pixel. The original grayscale value in the image frame to be processed is replaced with the corrected grayscale value of each pixel to obtain the corrected image frame. Using the first image frame as the first frame of the sequence, the corrected image frames are arranged sequentially after the first frame of the sequence according to the original acquisition time sequence corresponding to each corrected image frame, thus generating a normalized image sequence.

3. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The method involves jointly extracting temporally stable and spatially consistent components from a normalized image sequence, marking temporally unstable but spatially consistent components as time-varying interference masks, and marking temporally stable and spatially consistent components as stable structural response bases. Local gain suppression is then applied to the stable structural response bases based on the time-varying interference masks to obtain the suppressed structural response bases, including: A gray-scale temporal sequence is constructed from the gray values ​​of pixels with the same spatial coordinates in each frame of the normalized image sequence. The standard deviation of the gray values ​​in the gray-scale temporal sequence is used as the temporal fluctuation value. Pixels with temporal fluctuation values ​​less than a preset temporal fluctuation threshold are marked as temporally stable positions, and pixels with temporal fluctuation values ​​greater than or equal to the preset temporal fluctuation threshold are marked as temporally unstable positions. Calculate the phase consistency value pixel by pixel for the last frame of the normalized image sequence to form a spatial phase consistency value map. Mark the pixel positions in the spatial phase consistency value map that have a phase consistency value greater than a preset spatial consistency threshold as spatial consistency positions. The pixel positions that are simultaneously marked as temporally unstable positions and spatially consistent positions are marked as time-varying interference positions, and a time-varying interference mask is constructed with the pixel values ​​of all time-varying interference positions being 1. The pixel positions that are simultaneously marked as stable positions in the temporal domain and consistent positions in the spatial domain are marked as stable structure positions. The phase consistency values ​​at each stable structure position in the spatial phase consistency value map are used as the basis response amplitudes to generate a stable structure response basis. Multiply the amplitude of the base response at the location corresponding to the pixel value of the time-varying interference mask in the stable structural response base by a preset suppression coefficient to obtain the suppressed structural response base.

4. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The step of calculating the phase consistency value pixel by pixel in the last frame of the normalized image sequence to construct a spatial phase consistency value map includes: Construct a Log-Gabor filter, and convolve the Log-Gabor filter at each scale with the last frame image to obtain the even-symmetric and odd-symmetric filtering responses of each pixel at each scale. Perform arctangent operation on the even-symmetric and odd-symmetric filter responses of each pixel at each scale to obtain the local phase angle of each pixel at each scale; The local amplitude value of each pixel at each scale is obtained by summing the squares of the even-symmetric and odd-symmetric filter responses of each pixel at each scale and taking the square root. The sum of the local amplitude values ​​at all scales for each pixel is obtained. Summing the even-symmetric filter responses of each pixel at each scale yields the sum of even-symmetric responses; summing the odd-symmetric filter responses of each pixel at each scale yields the sum of odd-symmetric responses; performing an arctangent operation on the sum of even-symmetric responses and the sum of odd-symmetric responses yields the combined phase angle. Perform a difference operation on the local phase angle and the overall phase angle of each pixel at each scale to obtain the phase deviation angle; perform a cosine operation on the phase deviation angle to obtain the phase deviation cosine value; perform a multiplication operation on the phase deviation cosine value at each scale and the local amplitude value at the same scale and sum them to obtain the phase consistency numerator; divide the phase consistency numerator by the sum of the local amplitudes to obtain the phase consistency value. A spatial phase consistency value map is constructed using the phase consistency values ​​of all pixel locations.

5. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The process involves calculating the response intensity distribution and principal direction distribution based on the suppressed structural response baseline, segmenting the last frame corresponding to the suppressed structural response baseline into sub-regions, calculating the matching degree based on the local peaks in the response intensity distribution and the principal peaks in the principal direction distribution of each sub-region, and generating structural confidence based on the matching degree, including: By performing convolution operations on the suppressed structural response basis using the Sobel operator, the horizontal and vertical gradient values ​​of each pixel are obtained. Perform arctangent operation on the gradient values ​​in the horizontal and vertical directions to obtain the gradient direction angle; The gradient magnitude is obtained by summing the squares of the horizontal and vertical gradient values ​​and then taking the square root. The response intensity distribution map is constructed using the gradient magnitude of all pixels, and the main direction distribution map is constructed using the gradient direction angle of all pixels. Divide the last frame image corresponding to the suppressed structural response basis into sub-regions; determine the angle interval according to the distribution range of all pixel gradient direction angles in the main direction distribution map, and divide the gradient direction interval according to the angle interval; The number of pixels falling into each gradient direction interval is counted in each sub-region, and the angle value corresponding to the gradient direction interval with the most pixels is taken as the main direction and main peak of that sub-region. Within each sub-region, the pixel location in the response intensity distribution map whose gradient magnitude is greater than the gradient magnitude of all pixels within the neighborhood radius is searched and identified as the local peak intensity. The neighborhood radius is determined based on the standard deviation of the gradient magnitude within the sub-region. Calculate the ratio of the number of pixels corresponding to the main peak in the main direction in each sub-region to the total number of pixels in the sub-region, and mark the ratio as the concentration of the main direction in that sub-region. Calculate the ratio of the gradient magnitude of the local intensity peak in each sub-region to the average gradient magnitude of all pixels in the sub-region, and label the ratio as the intensity saliency of that sub-region; multiply the principal direction concentration and the intensity saliency to obtain the matching degree; Sub-regions with a matching degree greater than a preset matching degree threshold are marked as high-confidence sub-regions, and sub-regions with a matching degree less than or equal to the preset matching degree threshold are marked as low-confidence sub-regions, thus forming the structural confidence.

6. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The preset prior model extracts the node coordinates and adjacency relationships of the track centerline from the track design drawings, calculates and stores the node spacing based on the adjacent node coordinates, and includes track node sequences, node adjacency relationships, and node spacing attributes. In low-confidence sub-regions where the structural confidence level is below a preset confidence threshold, the local peak search and connection of the suppressed structural response basis are guided by the local orientation of the preset prior model to form a compensated structural segment. This compensated structural segment constraint updates the prior model, resulting in an updated spatial model and generating the compensated structural response basis, including: Obtain the orbit node sequence of the low-confidence sub-region in the prior model, perform linear interpolation on the orbit node sequence, and obtain the local orientation direction of each low-confidence sub-region; Within each low-confidence sub-region, along the local orientation, search for pixel locations on the suppressed structural response substrate where the response intensity is greater than a preset peak threshold to obtain the local peak location set for each low-confidence sub-region. According to the spatial order of the local orientation direction in each low-confidence sub-region, the local peak position sets of each low-confidence sub-region are connected sequentially to form a compensation structure segment; Obtain the node spacing of the prior model, and resample the compensation structure segment at equal intervals according to the node spacing to obtain the coordinates of the resampled nodes; The coordinates of the resampled nodes are compared with the coordinates of the corresponding orbital nodes in the prior model to obtain the position offset of each node. The coordinates of the corresponding nodes in the prior model are then updated with the position offset to obtain the updated spatial model. Extract the node coordinates located in the low-confidence sub-region in the updated spatial model, perform interpolation on each node coordinate to obtain the reconstructed response value of each pixel position in the low-confidence sub-region, and replace the original response value of the corresponding pixel position in the suppressed structural response basis with the reconstructed response value to generate the compensated structural response basis.

7. The content presentation control method for a multimodal interactive trackbook according to claim 6, characterized in that, Within each low-confidence sub-region, pixel locations with response intensities greater than a preset peak threshold are searched along the local directional direction on the suppressed structural response substrate to obtain a set of local peak locations for each low-confidence sub-region, including: Using the local orientation direction as the search axis direction, the suppressed structural response substrate is sampled at intervals along the search axis direction to obtain the sampling position sequence. The response intensity values ​​of all pixels are counted in the direction perpendicular to the search axis. The sum of the mean and standard deviation of the response intensity values ​​is calculated and used as the threshold for determining local peak values. Pixels whose response intensity values ​​within the window are greater than the local peak determination threshold are marked as candidate peak locations; The candidate peak positions adjacent in the search axis direction are merged into the same peak cluster according to the spatial adjacency relationship. The pixel position corresponding to the maximum value of the response intensity value of all candidate peak positions in the peak cluster is calculated, and the pixel position is taken as the local peak position of the peak cluster. The local peak positions of all peak clusters within each low-confidence sub-region are collected to generate a set of local peak positions.

8. The content presentation control method for a multimodal interactive trackbook according to claim 1, characterized in that, The process of performing peak filtering and threshold connection on the compensated structural response basis to obtain a stable structural binary map, and calculating the geometric deviation between the stable structural binary map and the updated spatial model, includes: Obtain the response intensity value and gradient direction angle in the principal direction distribution diagram of the compensated structural response base, and perform non-maximum suppression on the compensated structural response base along the gradient direction angle to obtain the non-maximum suppressed response base. By performing double-threshold hysteresis boundary connections on the response basis after nonmaximum suppression, a stable binary structure graph is obtained. Extract the edge point set from the binary graph of the stable structure, extract the node coordinates of the updated spatial model as the model point set, perform iterative nearest point registration on the edge point set and the model point set to obtain the rotation matrix and translation vector; Perform coordinate transformation on the model point set using rotation matrix and translation vector to obtain the transformed model point set. Calculate the mean Euclidean distance between all point pairs of the transformed model point set and the edge point set, and mark the mean Euclidean distance as the geometric deviation.

9. The content presentation control method for a multimodal interactive trackbook according to claim 8, characterized in that, When the geometric deviation exceeds a preset deviation threshold, the tactile array is driven to generate a directional vibration sequence. The feedback timescale of the directional vibration sequence, the timescale generated from the binary image of the stable structure, and the spatial distribution vector of the deviation are fused into a track tracking state vector. Control commands are presented based on the parsed content of the track tracking state vector, including: The geometric deviation is compared with a preset deviation threshold. If the geometric deviation is greater than the preset deviation threshold, the tactile array is driven to generate a directional vibration sequence, and the generation time of the directional vibration sequence is recorded as a feedback time stamp. The generation time of the binary image of the stable structure is obtained as the generation timescale; Extract all point pairs of Euclidean distances between the edge point set and the transformed model point set, and use all point pairs of Euclidean distances to form the spatial distribution vector of the deviation; The spatial distribution vectors of the feedback timescale, the generation timescale, and the deviation are concatenated to obtain the track tracking status vector. Based on the track tracking state vector, a preset content presentation control command library is retrieved, and the content presentation control command that matches the track tracking state vector is output.

10. A content presentation control system for a multimodal interactive trackbook, used in accordance with the content presentation control method for a multimodal interactive trackbook as described in any one of claims 1 to 9, characterized in that, include: Image acquisition module: used to acquire the image sequence of the track area on the current page of the track book, and perform inter-frame illumination normalization to obtain a normalized image sequence; Interference suppression module: used to jointly extract temporally stable components and spatially consistent components from normalized image sequences, mark temporally unstable but spatially consistent components as time-varying interference masks, mark temporally stable and spatially consistent components as stable structural response bases, and perform local gain suppression on stable structural response bases based on time-varying interference masks to obtain suppressed structural response bases. Confidence assessment module: used to calculate the response intensity distribution and main direction distribution based on the suppressed structural response base, divide the last frame corresponding to the suppressed structural response base into sub-regions, calculate the matching degree based on the local peak in the response intensity distribution and the main peak in the main direction distribution of each sub-region, and generate structural confidence based on the matching degree; Model update module: Used to search for and connect local peaks of the suppressed structural response basis in low-confidence sub-regions where the structural confidence is lower than a preset confidence threshold, guided by the local orientation of the preset prior model, to form a compensated structural segment. The prior model is then updated with the constraints of the compensated structural segment to obtain the updated spatial model and generate the compensated structural response basis. Deviation quantization module: used to perform peak filtering and threshold connection on the compensated structural response basis to obtain a stable structure binary map, and calculate the geometric deviation between the stable structure binary map and the updated spatial model; State analysis module: When the geometric deviation exceeds the preset deviation threshold, it drives the tactile array to generate a directional vibration sequence. It integrates the feedback timescale of the directional vibration sequence, the timescale generated by the binary map of the stable structure, and the spatial distribution vector of the deviation into a track tracking state vector, and presents control commands based on the analysis content of the track tracking state vector.

Images