A dynamic information location determination method

Abstract

The application discloses a dynamic information position determination method, which comprises the following steps: obtaining document object model and visual layout feature information of a target interactive area, and recording an interactive behavior sequence of a user on the target interactive area; performing multi-modal fusion analysis on the document object model, the visual layout feature information and the interactive behavior sequence to obtain a user intention classification result, and selecting target dynamic information in a preset dynamic information library according to the user intention classification result; performing visual saliency analysis on the visual layout feature information to obtain a visual center area, predicting a target display position based on the visual center area and the interactive behavior sequence, dynamically adjusting the document object model according to the target display position, and generating a dynamic information container. According to the technical scheme, the interactive behavior of a user and the characteristics of a target interactive area are analyzed by fusion, the personalized matching and dynamic optimization of dynamic information display content and display position are realized, and the dynamic information reach rate is improved.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a method for determining the location of dynamic information. Background Technology

[0002] Dynamic information, as an important carrier of digital content dissemination, directly impacts the overall value of information dissemination through its reach and conversion efficiency. Embedding dynamic information into digital content display carriers such as web pages and application interfaces can effectively reach and convert users' information.

[0003] Current methods for displaying dynamic information typically involve statically displaying it at a fixed location within the target interactive area. This means the display area and time period for the dynamic information are pre-defined in the implementation code of the target interactive area, and the same dynamic information is pushed to all users accessing that area. This approach is prone to low reach rates due to misalignment between the dynamic information's location and the user's visual focus. Furthermore, the same dynamic information cannot match the interests and needs of different users, further lowering the reach and conversion rates. Summary of the Invention

[0004] This application provides a method for determining the location of dynamic information. The purpose is to achieve personalized matching and dynamic optimization of the display content and location of dynamic information by integrating and analyzing user interaction behavior and target interaction area characteristics, thereby improving the reach of dynamic information.

[0005] In a first aspect, embodiments of this application provide a method for determining the location of dynamic information, the method comprising:

[0006] Obtain the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area;

[0007] Multimodal fusion analysis is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain user intent classification results, and target dynamic information is selected from a preset dynamic information database based on the user intent classification results;

[0008] Visual saliency analysis is performed on the visual layout feature information to obtain the visual center region. Based on the visual center region and the interaction behavior sequence, the target display position is predicted. The document object model is dynamically adjusted according to the target display position to generate a dynamic information container. The target dynamic information is rendered and displayed in the dynamic information container.

[0009] Optionally, the step of performing multimodal fusion analysis on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain the user intent classification result includes:

[0010] Feature extraction is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain document structure features, visual layout features, and interaction behavior features;

[0011] Semantic alignment is performed on the document structure features, the visual layout features, and the interactive behavior features to obtain a cross-modal unified semantic feature vector;

[0012] The cross-modal unified semantic feature vector is input into a pre-trained intent classification model to obtain the user intent classification result output by the intent classification model; wherein, the intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss and dynamic information transformation association loss.

[0013] Optionally, the step of semantically aligning the document structure features, the visual layout features, and the interaction behavior features to obtain a cross-modal unified semantic feature vector includes:

[0014] The document structure features, the visual layout features, and the interactive behavior features are respectively mapped to a shared latent semantic space;

[0015] In the shared latent semantic space, the attention weights of the visual layout features and the interactive behavior features relative to the document structure features are calculated;

[0016] Based on the attention weights, the visual layout features and the interactive behavior features are enhanced to obtain enhanced visual features and enhanced behavior features.

[0017] The document structure features, the enhanced visual features, and the enhanced behavioral features are concatenated, and the concatenation result is reduced in dimensionality and integrated to obtain a cross-modal unified semantic feature vector.

[0018] Optionally, predicting the target display location based on the visual center region and the interaction behavior sequence includes:

[0019] The target interactive area is divided into multiple display areas, and the visual center area and the interactive positions corresponding to each interactive behavior in the interactive behavior sequence are mapped to the corresponding display areas to obtain a display area sequence.

[0020] Based on the transition path information between adjacent display areas in the display area sequence, the state transition probability of the multiple display areas is calculated to obtain a state transition probability matrix.

[0021] The target display location is predicted based on the state transition probability matrix.

[0022] Optionally, the step of calculating the state transition probabilities of the multiple display areas based on the transition path information between adjacent display areas in the display area sequence to obtain a state transition probability matrix includes:

[0023] The assignment weights corresponding to the transfer path information are determined based on the interactive behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence.

[0024] For each display area, the weighted number of transfers from the display area to other display areas is calculated based on the transfer path information and the assigned weights.

[0025] Based on the weighted number of transitions from the display area to other display areas, the state transition probability of the display area to other display areas is determined, and a state transition probability matrix is ​​obtained.

[0026] Optionally, predicting the target display location based on the state transition probability matrix includes:

[0027] The display area corresponding to the current interactive behavior is obtained as the first initial display area, and the display area corresponding to the visual center area is obtained as the second initial display area.

[0028] Based on the state transition probability matrix, the cumulative arrival probability of each display area within a preset number of transitions is calculated, starting from the first initial display area and the second initial display area.

[0029] The cumulative arrival probability of the target for each display area is obtained by weighted summation of the two cumulative arrival probabilities of each display area.

[0030] The target display location is determined based on the display area with the highest cumulative arrival probability of the target.

[0031] Optionally, determining the target display location based on the display area with the highest cumulative arrival probability of the target includes:

[0032] Based on the visual layout feature information and the preset dynamic information container size, at least one blank sub-area is selected within the display area where the cumulative arrival probability of the target is highest.

[0033] Calculate the distance information between the blank sub-region and the visual center region, and determine the peripheral visual interference coefficient of the blank sub-region based on the visual layout feature information;

[0034] The distance information and the surrounding visual interference coefficient are weighted and summed to obtain the comprehensive score of the dynamic information display of the blank sub-region;

[0035] The blank sub-area with the highest overall score for the dynamic information display is determined as the target display location.

[0036] Secondly, embodiments of this application provide a dynamic information location determination device, the device comprising:

[0037] The feature acquisition module is used to acquire the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area;

[0038] The information selection module is used to perform multimodal fusion analysis on the document object model, the visual layout feature information and the interaction behavior sequence to obtain the user intent classification result, and select target dynamic information from the preset dynamic information database based on the user intent classification result;

[0039] The location determination module is used to perform visual saliency analysis on the visual layout feature information to obtain the visual center region, and predict the target display position based on the visual center region and the interaction behavior sequence. It is used to dynamically adjust the document object model according to the target display position and generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

[0040] Optionally, the information selection module is specifically used for:

[0041] Feature extraction is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain document structure features, visual layout features, and interaction behavior features;

[0042] Semantic alignment is performed on the document structure features, the visual layout features, and the interactive behavior features to obtain a cross-modal unified semantic feature vector;

[0043] The cross-modal unified semantic feature vector is input into a pre-trained intent classification model to obtain the user intent classification result output by the intent classification model; wherein, the intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss and dynamic information transformation association loss.

[0044] Optionally, the information selection module is specifically used for:

[0045] The document structure features, the visual layout features, and the interactive behavior features are respectively mapped to a shared latent semantic space;

[0046] In the shared latent semantic space, the attention weights of the visual layout features and the interactive behavior features relative to the document structure features are calculated;

[0047] Based on the attention weights, the visual layout features and the interactive behavior features are enhanced to obtain enhanced visual features and enhanced behavior features.

[0048] The document structure features, the enhanced visual features, and the enhanced behavioral features are concatenated, and the concatenation result is reduced in dimensionality and integrated to obtain a cross-modal unified semantic feature vector.

[0049] Optionally, the position determination module is specifically used for:

[0050] The target interactive area is divided into multiple display areas, and the visual center area and the interactive positions corresponding to each interactive behavior in the interactive behavior sequence are mapped to the corresponding display areas to obtain a display area sequence.

[0051] Based on the transition path information between adjacent display areas in the display area sequence, the state transition probability of the multiple display areas is calculated to obtain a state transition probability matrix.

[0052] The target display location is predicted based on the state transition probability matrix.

[0053] Optionally, the position determination module is specifically used for:

[0054] The assignment weights corresponding to the transfer path information are determined based on the interactive behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence.

[0055] For each display area, the weighted number of transfers from the display area to other display areas is calculated based on the transfer path information and the assigned weights.

[0056] Based on the weighted number of transitions from the display area to other display areas, the state transition probability of the display area to other display areas is determined, and a state transition probability matrix is ​​obtained.

[0057] Optionally, the position determination module is specifically used for:

[0058] The display area corresponding to the current interactive behavior is obtained as the first initial display area, and the display area corresponding to the visual center area is obtained as the second initial display area.

[0059] Based on the state transition probability matrix, the cumulative arrival probability of each display area within a preset number of transitions is calculated, starting from the first initial display area and the second initial display area.

[0060] The cumulative arrival probability of the target for each display area is obtained by weighted summation of the two cumulative arrival probabilities of each display area.

[0061] The target display location is determined based on the display area with the highest cumulative arrival probability of the target.

[0062] Optionally, the position determination module is specifically used for:

[0063] Based on the visual layout feature information and the preset dynamic information container size, at least one blank sub-area is selected within the display area where the cumulative arrival probability of the target is highest.

[0064] Calculate the distance information between the blank sub-region and the visual center region, and determine the peripheral visual interference coefficient of the blank sub-region based on the visual layout feature information;

[0065] The distance information and the surrounding visual interference coefficient are weighted and summed to obtain the comprehensive score of the dynamic information display of the blank sub-region;

[0066] The blank sub-area with the highest overall score for the dynamic information display is determined as the target display location.

[0067] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method described in the first aspect.

[0068] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0069] In this embodiment, a document object model and visual layout feature information of the target interaction area are obtained, and a sequence of user interaction behaviors on the target interaction area is recorded. Multimodal fusion analysis is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain a user intent classification result. Based on the user intent classification result, target dynamic information is selected from a preset dynamic information database. Visual saliency analysis is performed on the visual layout feature information to obtain a visual center region. Based on the visual center region and the interaction behavior sequence, a target display position is predicted, which is used to dynamically adjust the document object model according to the target display position and generate a dynamic information container. The target dynamic information is then rendered and displayed in the dynamic information container. This dynamic information position determination method, by fusing and analyzing user interaction behaviors and target interaction area features, achieves personalized matching and dynamic optimization of dynamic information display content and display position, thereby improving the dynamic information reach rate. Attached Figure Description

[0070] Figure 1This is a flowchart illustrating a dynamic information location determination method provided in an embodiment of this application;

[0071] Figure 2 This is a flowchart illustrating another dynamic information location determination method provided in an embodiment of this application;

[0072] Figure 3 This is a flowchart illustrating another dynamic information location determination method provided in the embodiments of this application;

[0073] Figure 4 This is an example diagram illustrating the calculation of cumulative arrival probability provided in the embodiments of this application;

[0074] Figure 5 This is a schematic diagram of the structure of a dynamic information location determination device provided in an embodiment of this application;

[0075] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0077] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0078] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0079] The dynamic information location determination method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0080] First, this application is applicable to scenarios where precise dynamic information is displayed in target interactive areas within Internet web pages and mobile applications, especially to personalized marketing scenarios where the position and content of dynamic information need to be dynamically adjusted based on real-time user interaction behavior. It is understood that the executing entity of this application may be a server, server cluster, or terminal device, without specific limitations here.

[0081] Figure 1 This is a flowchart illustrating a dynamic information location determination method provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following:

[0082] S101, Obtain the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area.

[0083] Dynamic information can be any type of information that is disseminated through digital carriers, has real-time update capabilities, and needs to be displayed in a targeted manner on the interactive interface, including but not limited to service recommendation information, content push information, function guidance information, and contextual prompts.

[0084] The target interactive area can be a digital interface area where users can perform human-computer interaction operations. Specifically, it can be reflected in the interactive functional module area in a webpage, the independent operation display area in an application interface, etc. The target interactive area has the ability to respond to user interactive operations such as clicking, swiping, and hovering.

[0085] The Document Object Model (DOM) can be a programming interface for representing and manipulating markup language documents such as HTML (HyperText Markup Language) and XML (eXtensible Markup Language). It abstracts the document structure into a node tree, where each node corresponds to an element, attribute, or text content in the document structure, and supports dynamic operations such as adding, deleting, modifying, and querying page elements.

[0086] In one embodiment, the document object model of the target interactive area can be obtained by calling the document object interface through JavaScript to traverse the page node tree, collect the tag type, attribute information, hierarchical relationship and position parameters of each node, or by using the DOM (Document Object Model) parsing API (Application Programming Interface) provided by the browser kernel of the browser where the target interactive area is located, directly exporting the structured DOM data of the target interactive area and storing it as a JSON format file.

[0087] The visual layout feature information can be a collection of spatial distribution features and attribute features of each visual element in the target interactive area, including but not limited to the size, position coordinates, color contrast, shape features, hierarchical relationship, spacing and alignment between visual elements, etc.

[0088] In one embodiment, the visual layout feature information of the target interactive area can be obtained by taking a screenshot of the rendered target interactive area and extracting features from the screenshot to obtain the visual layout feature information. For example, edge detection algorithms can be used to identify the boundaries of visual elements, and color histograms can be used to analyze the color contrast of visual elements.

[0089] The user's interaction sequence on the target interaction area can be an ordered set of continuous interactive actions generated by the user within a preset time window after the target interaction area is rendered. These actions include, but are not limited to, clicks, swipes, dwell time, page zooming, and element dragging. Each action includes attributes such as action type, trigger timestamp, and element identifier.

[0090] In one embodiment, the method of recording the user's interaction behavior sequence in the target interaction area can be to embed a tracking script in the target interaction area, listen to the user's mouse events, touch events and keyboard events, collect the behavior type, trigger timestamp, behavior duration and active element identifier of each interaction behavior in real time, and serialize and store them in chronological order to obtain the user's interaction behavior sequence in the target interaction area.

[0091] S102, perform multimodal fusion analysis on the document object model, the visual layout feature information and the interaction behavior sequence to obtain the user intent classification result, and select target dynamic information from the preset dynamic information database based on the user intent classification result.

[0092] The user intent classification result can be the result of dividing the user's current intent to access the target interaction area into preset categories, which include, but are not limited to, information retrieval intent, product purchase intent, function usage intent, leisure and entertainment intent, etc.

[0093] In one embodiment, the method of obtaining user intent classification results by performing multimodal fusion analysis on document object models, visual layout feature information, and interaction behavior sequences can be achieved by pre-constructing a mapping relationship between the document object model, visual layout feature information, interaction behavior sequences, and user intent classification results. Using the current document object model, visual layout feature information, and interaction behavior sequences as query conditions, the mapping relationship is queried, and the resulting query results include the user intent classification results. As an example, the document object model contains numerous product card nodes with node attributes including price and a purchase button; the visual layout feature information includes vibrant colors, prominent promotional labels, and a large image area; and the interaction behavior sequence involves multiple rapid swipes followed by stopping at a product card area and clicking to view details. The corresponding user intent classification result is a product purchase intent.

[0094] The preset dynamic information database can be a pre-built structured database containing massive amounts of dynamic information and related attribute information; the target dynamic information can be dynamic information selected from the preset dynamic information database that highly matches the current user intent classification result.

[0095] In one embodiment, the method of selecting target dynamic information from a preset dynamic information database based on user intent classification results can be achieved by using the current user intent classification results as query conditions to query the preset dynamic information database to obtain candidate dynamic information, acquiring the content theme or context information of the target interaction area, and selecting a preset number of candidate dynamic information with the highest similarity between information features and content theme or context information as target dynamic information.

[0096] S103, perform visual saliency analysis on the visual layout feature information to obtain the visual center region, and predict the target display position based on the visual center region and the interaction behavior sequence, so as to dynamically adjust the document object model according to the target display position and generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

[0097] Among them, the visual center area can be the area that is most likely to attract the user's visual attention in the target interaction area.

[0098] In one embodiment, the method of performing visual saliency analysis on visual layout feature information to obtain the visual center region can be to construct a visual attention graph by using each visual element in the target interaction area as a node of the graph, and the spatial distance and color similarity between visual elements as edge weights. The centrality score of each node in the visual attention graph is calculated, and the spatial range of the visual element corresponding to one or more nodes with the highest centrality score in the target interaction area is determined as the visual center region.

[0099] The target display location can be determined by comprehensively considering the visual center area and user interaction habits, and is the most suitable location to display the target's dynamic information.

[0100] In one embodiment, the method of predicting the target display location based on the visual center region and the interaction behavior sequence can be achieved by statistically analyzing the user's click hotspots and swipe dwell areas in the interaction behavior sequence and identifying them as the user's high-frequency interaction areas, and calculating the intersection area of ​​the visual center region and the user's high-frequency interaction areas as the target display location.

[0101] Among them, the dynamic information container can be an independent DOM node container generated based on the adjusted document object model, used to carry and render target dynamic information. The dynamic information container can be used to render and display target dynamic information. Specifically, the dynamic information container has a built-in information rendering engine that supports the parsing and display of various information formats.

[0102] In one embodiment, the method of dynamically adjusting the document object model and generating a dynamic information container according to the target display position can be achieved by calling the DOM manipulation API, inserting a new DOM node as a dynamic information container under the parent node corresponding to the target display position, setting style attributes such as size, position, and hierarchy of the dynamic information container to match the target display position, and passing the target dynamic information into the rendering engine of the dynamic information container to complete the visualization display of the target dynamic information.

[0103] In this embodiment, a document object model and visual layout feature information of the target interaction area are obtained, and a sequence of user interaction behaviors on the target interaction area is recorded. Multimodal fusion analysis is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain a user intent classification result. Based on the user intent classification result, target dynamic information is selected from a preset dynamic information database. Visual saliency analysis is performed on the visual layout feature information to obtain a visual center region. Based on the visual center region and the interaction behavior sequence, a target display position is predicted, which is used to dynamically adjust the document object model according to the target display position and generate a dynamic information container. The target dynamic information is then rendered and displayed in the dynamic information container. This dynamic information position determination method, by fusing and analyzing user interaction behaviors and target interaction area features, achieves personalized matching and dynamic optimization of dynamic information display content and display position, thereby improving the dynamic information reach rate.

[0104] Figure 2 This is a flowchart illustrating another dynamic information location determination method provided in an embodiment of this application. For example... Figure 2 As shown, the specific steps include the following:

[0105] S201, Obtain the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area.

[0106] S202, feature extraction is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain document structure features, visual layout features, and interaction behavior features.

[0107] Among them, document structure features can be quantitative features extracted from the node tree of the document object model to represent the semantics and organizational logic of page content; visual layout features can be numerical vectors obtained by encoding visual layout feature information; and interactive behavior features can be statistical and sequential features obtained by structuring interactive behavior sequences.

[0108] In one embodiment, feature extraction is performed on the document object model, visual layout features, and interaction behavior sequences to obtain document structure features, visual layout features, and interaction behavior features. This can be achieved by serializing the document object model and inputting it into a Transformer-based text encoder or graph neural network to obtain document structure features; performing deep feature extraction and aggregation on the visual layout features using an object detection model, deep convolutional neural network, or visual Transformer to obtain visual layout features; and converting the interaction behavior sequences into event sequences containing time, location, and type, extracting their dynamic patterns using a time-series model to form interaction behavior features.

[0109] S203, semantically align the document structure features, the visual layout features, and the interactive behavior features to obtain a cross-modal unified semantic feature vector.

[0110] Among them, the cross-modal unified semantic feature vector can be a low-dimensional dense vector representation that integrates information from three modalities: document, visual, and interaction.

[0111] In one embodiment, semantic alignment of document structure features, visual layout features, and interactive behavior features to obtain a cross-modal unified semantic feature vector can be achieved by using a Transformer-based multimodal encoder to perform deep feature interaction and fusion of document structure features, visual layout features, and interactive behavior features, ultimately outputting a cross-modal semantic feature vector.

[0112] In one embodiment, semantically aligning the document structure features, visual layout features, and interactive behavior features to obtain a cross-modal unified semantic feature vector includes: mapping the document structure features, visual layout features, and interactive behavior features to a shared latent semantic space; calculating the attention weights of the visual layout features and interactive behavior features relative to the document structure features in the shared latent semantic space; performing feature enhancement on the visual layout features and interactive behavior features according to the attention weights to obtain enhanced visual features and enhanced behavior features; concatenating the document structure features, enhanced visual features, and enhanced behavior features, and performing dimensionality reduction and integration on the concatenation result to obtain a cross-modal unified semantic feature vector.

[0113] In this context, the shared latent semantic space can be a vector space of a predefined dimension, within which feature vectors from different modalities possess comparable semantic metrics.

[0114] In one embodiment, the method of mapping document structure features, visual layout features, and interactive behavior features to a shared latent semantic space can be achieved by configuring an independent trainable linear projection layer (fully connected layer) for each of the document structure features, visual layout features, and interactive behavior features, so as to uniformly project the dimensions of the document structure features, visual layout features, and interactive behavior features to a preset dimension of the shared latent semantic space.

[0115] The attention weight can be a scalar value or a vector, used to quantify the degree to which each dimension or the whole of the visual layout features / interaction behavior features are semantically related to the document structure features in the user intent classification result recognition task.

[0116] In one embodiment, the attention weights of visual layout features and interactive behavior features relative to document structure features in a shared latent semantic space can be obtained by using document structure features as query vector Q, converting visual layout features / interactive behavior features into key vector K and value vector V respectively, calculating the dot product of Q and K and performing scaling and Softmax normalization to obtain the attention weights.

[0117] Among them, enhanced visual features can be the feature vector obtained by weighting the original visual layout features with the attention weights of the document structure features; enhanced behavioral features can be the feature vector obtained by weighting the original interactive behavioral features with the attention weights of the document structure features.

[0118] In one embodiment, visual layout features and interactive behavior features are enhanced according to attention weights to obtain enhanced visual features and enhanced behavior features. This can be achieved by using an attention weighted fusion algorithm, which weights and combines each dimension of the original visual layout feature vector / interactive behavior feature vector based on attention weights to obtain enhanced visual features / enhanced behavior features.

[0119] In one embodiment, document structure features, enhanced visual features, and enhanced behavioral features are concatenated, and the concatenation result is reduced in dimensionality and integrated to obtain a cross-modal unified semantic feature vector. This can be achieved by concatenating the document structure features, enhanced visual features, and enhanced behavioral features along their feature dimensions to obtain a higher-dimensional joint vector. This joint vector is then input into one or more fully connected layers, ultimately outputting a fixed-dimensional cross-modal unified semantic feature vector. The one or more fully connected layers can constitute a Multilayer Perceptron (MLP).

[0120] The advantage of this approach is that by constructing a shared implicit semantic space and utilizing an attention mechanism that uses document structure as the query, it can effectively use the core content of the page as an anchor point to dynamically filter and enhance the most relevant visual presentation information and user interaction patterns, thereby achieving guided and refined multimodal semantic alignment.

[0121] S204, the cross-modal unified semantic feature vector is input into the pre-trained intent classification model to obtain the user intent classification result output by the intent classification model; wherein, the intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss and dynamic information transformation association loss.

[0122] The intent classification model can be a discriminative model that takes a unified cross-modal semantic feature vector as input and outputs the user intent classification result. Its structure can be a multilayer perceptron classifier head or a softmax classifier. The intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss, and dynamic information transformation association loss. Specifically, the intent classification loss can be a standard classification loss function, such as cross-entropy loss, used to measure the difference between the user intent category predicted by the model and the true labeled category; the cross-modal feature alignment loss can be a loss function used to constrain and optimize the feature alignment process, such as contrastive loss or triplet loss; and the dynamic information transformation association loss can be a supervisory signal incorporating business feedback, such as a loss based on click-through rate prediction.

[0123] In one embodiment, the intention classification model can be jointly optimized and trained based on intention classification loss, cross-modal feature alignment loss, and dynamic information transformation association loss. This can be achieved by calculating the intention classification loss, cross-modal feature alignment loss, and dynamic information transformation association loss simultaneously during the forward propagation phase of the intention classification model, and then linearly combining these losses according to their importance during backpropagation to obtain the total loss. The total loss is then optimized using the gradient descent algorithm to update the model parameters of the intention classification model end-to-end.

[0124] S205, Select target dynamic information from the preset dynamic information database based on the user intent classification result.

[0125] S206, Perform visual saliency analysis on the visual layout feature information to obtain the visual center region, and predict the target display position based on the visual center region and the interaction behavior sequence, so as to dynamically adjust the document object model according to the target display position and generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

[0126] The advantage of this approach is that by introducing multimodal feature extraction and semantic alignment techniques, it is possible to gain a more comprehensive and in-depth understanding of the content, form, and real-time user interaction of the target interaction area, thereby providing a more accurate insight into the user's true intent than single-modal analysis.

[0127] Figure 3 This is a flowchart illustrating another dynamic information location determination method provided in an embodiment of this application. For example... Figure 3 As shown, the specific steps include the following:

[0128] S301, Obtain the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area.

[0129] S302, perform multimodal fusion analysis on the document object model, the visual layout feature information and the interaction behavior sequence to obtain user intent classification results, and select target dynamic information from the preset dynamic information database based on the user intent classification results.

[0130] S303, Perform visual saliency analysis on the visual layout feature information to obtain the visual center region.

[0131] S304, the target interaction area is divided into multiple display areas, and the visual center area and the interaction positions corresponding to each interaction behavior in the interaction behavior sequence are mapped to the corresponding display areas to obtain the display area sequence.

[0132] The display area can be several logically or physically independent subspaces into which the entire target interactive area is divided.

[0133] In one embodiment, dividing the target interactive area into multiple display areas can be achieved by semantically segmenting the target interactive area based on the node hierarchy or visual layout features of the document object model to obtain the display areas. As an example, the method of semantically segmenting the target interactive area based on the node hierarchy of the document object model to obtain display areas can specifically involve aggregating spatially contiguous visual elements with the same parent node into a single display area.

[0134] The interactive position corresponding to the interactive behavior can be the specific location information of the point of action (such as mouse click coordinates, touch point coordinates) or the range of action (such as the area covered by the start and end coordinates of the sliding trajectory) in the page coordinate system of the target interactive area when the interactive behavior occurs.

[0135] The display area sequence can be a sequence of display area identifiers arranged in chronological order.

[0136] In one embodiment, the method of mapping the visual center region and the interaction positions corresponding to each interaction in the interaction sequence to the corresponding display areas to obtain the display area sequence can be achieved by matching the interaction positions corresponding to each interaction with each display area according to the interaction order of the interaction sequence to determine the display area corresponding to each interaction, sorting the display areas corresponding to each interaction according to the interaction order of the interaction sequence to obtain the display area sequence, and inserting the display area to which the center position of the visual center region belongs into the display area sequence as the first display area in the display area sequence.

[0137] S305, based on the transition path information between adjacent display areas in the display area sequence, calculate the state transition probability of the multiple display areas to obtain a state transition probability matrix.

[0138] Among them, the transfer path information between adjacent display areas can be the directional transfer relationship from the previous display area (source state) to the next display area (target state) in the display area sequence.

[0139] The state transition probability can be the likelihood of a user moving from one display area to another, usually represented by a value between 0 and 1. The state transition probability matrix can be an N×N square matrix (N is the total number of display areas), and the element value of the element in the i-th row and j-th column of the matrix represents the state transition probability of moving from the i-th display area to the j-th display area.

[0140] In one embodiment, the state transition probability matrix is ​​obtained by calculating the state transition probability of multiple display areas based on the transition path information between adjacent display areas in the display area sequence and the corresponding interactive behavior attribute information. This can be achieved by calculating the number of transitions from each display area to other display areas based on the transition path information, and then calculating the ratio of the number of transitions from each other display area to the total number of transitions from the display area to other display areas to obtain the state transition probability from the display area to other display areas, thus obtaining the state transition probability matrix.

[0141] In one embodiment, calculating the state transition probability of the plurality of display areas based on the transition path information between adjacent display areas in the display area sequence to obtain a state transition probability matrix includes: determining the assignment weight corresponding to the transition path information based on the interactive behavior attribute information corresponding to the transition path information between adjacent display areas in the display area sequence; for each display area, calculating the weighted number of transitions from the display area to other display areas based on the transition path information and the assignment weight; determining the state transition probability of the display area to other display areas based on the weighted number of transitions from the display area to other display areas to obtain a state transition probability matrix.

[0142] Among them, the interactive behavior attribute information corresponding to the transfer path information can be the attributes of the interactive behavior itself that triggers the area transfer, such as behavior type, behavior duration, etc.; the assigned weight can be a value calculated based on the interactive behavior attribute information, used to quantify the intensity of user attention migration or the clarity of intent reflected by the area transfer.

[0143] In one embodiment, the method of determining the assigned weight of transfer path information based on the interactive behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence can be achieved by pre-constructing a mapping relationship between behavior type, behavior duration, and assigned weight. The current behavior type and behavior duration are used as query conditions to query the mapping relationship, and the query result includes the assigned weight of the transfer path information. For example, if the current behavior type is a click and the current behavior duration is 3 seconds, then the assigned weight of the transfer path information is 0.5.

[0144] The weighted transfer count can be calculated by including the assigned weight corresponding to each transfer, rather than simply counting it as 1.

[0145] In one embodiment, for each display area, the weighted number of transfers from the display area to other display areas can be calculated based on the transfer path information and the assigned weights. This can be achieved by summing 1 with the assigned weights corresponding to the transfer path information to obtain the weighted number of transfers corresponding to the transfer path information, and summing the weighted number of transfers corresponding to all transfer path information of the display area to another display area to obtain the weighted number of transfers from the display area to that other display area.

[0146] In one embodiment, the state transition probability matrix is ​​obtained by determining the state transition probability of a display area to other display areas based on the weighted number of transitions from the display area to other display areas. This can be achieved by calculating the ratio of the weighted number of transitions from each display area to the total weighted number of transitions from the display area to other display areas, thus obtaining the state transition probability matrix.

[0147] The advantage of this approach is that by introducing weighted values ​​based on interactive behavior attributes, the simple frequency statistics of transitions are optimized into weighted statistics that can distinguish the quality of interactions. This makes the state transition probability matrix not only reflect the number of times user attention flows, but also the intensity of that flow, i.e. the degree of intent. This provides a more reliable basis for predicting the display location of high-potential dynamic information targets.

[0148] S306, predict the target display position based on the state transition probability matrix, and dynamically adjust the document object model according to the target display position to generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

[0149] In one embodiment, the method of predicting the target display location based on the state transition probability matrix can be achieved by obtaining the display area corresponding to the current interactive behavior, and determining other display areas whose state transition probability from the current display area to other display areas exceeds a preset state transition probability threshold as the target display location.

[0150] In one embodiment, predicting the target display location based on the state transition probability matrix includes: obtaining the display area corresponding to the current interaction behavior as a first initial display area, and the display area corresponding to the visual center area as a second initial display area; according to the state transition probability matrix, calculating the cumulative arrival probability of each display area within a preset number of transitions, starting from the first initial display area and the second initial display area respectively; performing a weighted summation of the two cumulative arrival probabilities of each display area to obtain the target cumulative arrival probability of each display area; and determining the target display location based on the display area with the highest target cumulative arrival probability.

[0151] The first initial display area can be the display area belonging to the location of the user's most recent interaction (i.e., the current interaction); the second initial display area is the display area belonging to the center of the visual center area.

[0152] The preset number of transfers can be a positive integer set based on experience or experimentation, used to limit the range or number of steps of predicting the user's future attention transfers; the cumulative arrival probability can be the sum of the probabilities of passing through a certain display area within the preset number of transfers, starting from a certain initial display area.

[0153] In one embodiment, based on the state transition probability matrix, the cumulative arrival probability of each display area within a preset number of transitions, starting from the first initial display area and the second initial display area, can be calculated. This can be achieved by selecting the current display area as the starting point of the current transition for each transition behavior, and selecting the other display area with the highest state transition probability corresponding to the current display area as the end point of the current transition behavior and the starting point of the next transition behavior. The probability of each display area being selected as the end point of the next transition behavior in each transition behavior is summed to obtain the cumulative arrival probability of each display area.

[0154] Figure 4 This is an example diagram illustrating the calculation of cumulative arrival probability provided in an embodiment of this application. For example... Figure 4As shown, the starting point of the first transition behavior is ① (corresponding to the 3rd display area), and the other display area with the highest state transition probability (0.3) (the 5th display area) is used as the endpoint of the first transition behavior ②; the starting point of the second transition behavior is ② (corresponding to the 5th display area), and the other display area with the highest state transition probability (0.8) (the 6th display area) is used as the endpoint of the second transition behavior ③; the starting point of the third transition behavior is ③ (corresponding to the 6th display area), and the other display area with the highest state transition probability (0.4) (the 2nd display area) is used as the endpoint of the third transition behavior ④; the starting point of the fourth transition behavior is ④ (corresponding to the 2nd display area), and the other display area with the highest state transition probability (0.6) (the 8th display area) is used as the endpoint of the fourth transition behavior ⑤; the starting point of the fifth transition behavior is ⑤ (corresponding to the 8th display area), and the other display area with the highest state transition probability (0.6) (the 1st display area) is used as the endpoint of the fifth transition behavior ⑥. Correspondingly, the cumulative arrival probabilities for each display area are 0.6, 0.4, 0, 0, 0.3, 0.8, 0, and 0.6, respectively.

[0155] The cumulative arrival probability of the target can be a comprehensive probability value obtained by linearly weighting and summing the two cumulative arrival probabilities calculated from the first initial display area and the second initial display area according to preset weights.

[0156] In one embodiment, the method of calculating the target cumulative arrival probability of each display area by weighted summation of the two cumulative arrival probabilities of each display area can be as follows: assign a weight α to the cumulative arrival probability corresponding to the first initial display area and a weight β to the cumulative arrival probability corresponding to the second initial display area. For each display area, multiply the cumulative arrival probability corresponding to the first initial display area by α and multiply the cumulative arrival probability corresponding to the second initial display area by β, and finally sum the two multiplication results to obtain the target cumulative arrival probability of the display area.

[0157] In one embodiment, the method of determining the target display location based on the display area with the highest cumulative arrival probability of the target can be to determine the target display location as the entire display area with the highest cumulative arrival probability of the target.

[0158] In one embodiment, determining the target display location based on the display area with the highest cumulative arrival probability of the target includes: selecting at least one blank sub-region within the display area with the highest cumulative arrival probability of the target according to the visual layout feature information and the preset dynamic information container size; calculating the distance information between the blank sub-region and the visual center region, and determining the surrounding visual interference coefficient of the blank sub-region according to the visual layout feature information; performing a weighted summation calculation on the distance information and the surrounding visual interference coefficient to obtain a comprehensive dynamic information display score for the blank sub-region; and determining the blank sub-region with the highest comprehensive dynamic information display score as the target display location.

[0159] The preset dynamic information container size can be the width and height values ​​of a predefined rectangular area that can hold dynamic information.

[0160] Among them, the blank sub-region can be a continuous available space area located inside the display area with the highest cumulative arrival probability of the target, whose space size is not less than the preset dynamic information container size, and which is not occupied or covered by page content elements such as text, images, and controls.

[0161] In one embodiment, the method of selecting at least one blank sub-region within the display area with the highest cumulative arrival probability of the target based on visual layout feature information and a preset dynamic information container size can be achieved by performing a sliding window scan within the coordinate range of the display area with the highest cumulative arrival probability of the target, setting the size of the scanning window to be equal to the preset dynamic information container size, calculating the intersection-union ratio (IUU) between each window position and existing page content elements based on visual layout feature information, and determining the area corresponding to the window position with an IUU of zero and completely contained within the display area with the highest cumulative arrival probability of the target as the blank sub-region.

[0162] The distance information between the blank sub-region and the visual center region can be the Euclidean distance between the center position of the blank sub-region and the center position of the visual center region.

[0163] Among them, the peripheral visual interference coefficient can be a quantitative value used to characterize the intensity of distraction or competition that other page visual elements may cause to the user's attention within a certain range around the blank sub-region.

[0164] In one embodiment, the method for determining the peripheral visual interference coefficient of a blank sub-region based on visual layout feature information can be as follows: an evaluation neighborhood is defined with the blank sub-region as the center, the visual saliency values ​​of all other visual elements in the evaluation neighborhood (which can be obtained through the visual saliency analysis described above) and their edge distances to the blank sub-region are obtained, the contribution of all neighborhood elements is calculated through a decay weighting function and summed, and the summation result is normalized to obtain the peripheral visual interference coefficient of the blank sub-region.

[0165] Among them, the dynamic information display comprehensive score can be a single quantitative indicator that combines two optimization objectives: distance information and surrounding visual interference coefficient.

[0166] In one embodiment, a weighted summation of distance information and surrounding visual interference coefficients is used to obtain the comprehensive score for the dynamic information display of the blank sub-region. This can be achieved by assigning a weight w1 to the distance information and a weight w2 to the surrounding visual interference coefficients. For each blank sub-region, the distance information is multiplied by w1, and the surrounding visual interference coefficient is multiplied by w2. Finally, the results of these two multiplications are summed to obtain the comprehensive score for the dynamic information display of the blank sub-region. Here, both w1 and w2 are negative numbers.

[0167] The advantage of this approach is that by balancing the two key factors of proximity to the visual focal point and minimizing environmental interference, it can select key locations that have both high exposure potential and ensure the clear and effective delivery of dynamic information content, thus achieving the best balance between user experience and dynamic information effectiveness.

[0168] The advantage of this approach is that it integrates both real-time user behavior prediction and static visual guidance on the page, making the predictions more comprehensive and robust.

[0169] The advantage of this approach is that by concretizing the abstract sequence of user interaction behaviors into migration trajectories between discrete display areas and constructing a quantified state transition probability matrix, the prediction of user attention flow and future interaction focus becomes more objective.

[0170] Figure 5 This is a schematic diagram of the structure of a dynamic information location determination device provided in an embodiment of this application. Figure 5 As shown, the device includes:

[0171] The feature acquisition module 510 is used to acquire the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area.

[0172] The information selection module 520 is used to perform multimodal fusion analysis on the document object model, the visual layout feature information and the interaction behavior sequence to obtain user intent classification results, and select target dynamic information from a preset dynamic information database based on the user intent classification results.

[0173] The position determination module 530 is used to perform visual saliency analysis on the visual layout feature information to obtain the visual center region, and predict the target display position based on the visual center region and the interaction behavior sequence. It is used to dynamically adjust the document object model according to the target display position and generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

[0174] Optionally, the information selection module 520 is specifically used for:

[0175] Feature extraction is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain document structure features, visual layout features, and interaction behavior features;

[0176] Semantic alignment is performed on the document structure features, the visual layout features, and the interactive behavior features to obtain a cross-modal unified semantic feature vector;

[0177] The cross-modal unified semantic feature vector is input into a pre-trained intent classification model to obtain the user intent classification result output by the intent classification model; wherein, the intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss and dynamic information transformation association loss.

[0178] Optionally, the information selection module 520 is specifically used for:

[0179] The document structure features, the visual layout features, and the interactive behavior features are respectively mapped to a shared latent semantic space;

[0180] In the shared latent semantic space, the attention weights of the visual layout features and the interactive behavior features relative to the document structure features are calculated;

[0181] Based on the attention weights, the visual layout features and the interactive behavior features are enhanced to obtain enhanced visual features and enhanced behavior features.

[0182] The document structure features, the enhanced visual features, and the enhanced behavioral features are concatenated, and the concatenation result is reduced in dimensionality and integrated to obtain a cross-modal unified semantic feature vector.

[0183] Optionally, the position determination module 530 is specifically used for:

[0184] The target interactive area is divided into multiple display areas, and the visual center area and the interactive positions corresponding to each interactive behavior in the interactive behavior sequence are mapped to the corresponding display areas to obtain a display area sequence.

[0185] Based on the transition path information between adjacent display areas in the display area sequence, the state transition probability of the multiple display areas is calculated to obtain a state transition probability matrix.

[0186] The target display location is predicted based on the state transition probability matrix.

[0187] Optionally, the position determination module 530 is specifically used for:

[0188] The assignment weights corresponding to the transfer path information are determined based on the interactive behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence.

[0189] For each display area, the weighted number of transfers from the display area to other display areas is calculated based on the transfer path information and the assigned weights.

[0190] Based on the weighted number of transitions from the display area to other display areas, the state transition probability of the display area to other display areas is determined, and a state transition probability matrix is ​​obtained.

[0191] Optionally, the position determination module 530 is specifically used for:

[0192] The display area corresponding to the current interactive behavior is obtained as the first initial display area, and the display area corresponding to the visual center area is obtained as the second initial display area.

[0193] Based on the state transition probability matrix, the cumulative arrival probability of each display area within a preset number of transitions is calculated, starting from the first initial display area and the second initial display area.

[0194] The cumulative arrival probability of the target for each display area is obtained by weighted summation of the two cumulative arrival probabilities of each display area.

[0195] The target display location is determined based on the display area with the highest cumulative arrival probability of the target.

[0196] Optionally, the position determination module 530 is specifically used for:

[0197] Based on the visual layout feature information and the preset dynamic information container size, at least one blank sub-area is selected within the display area where the cumulative arrival probability of the target is highest.

[0198] Calculate the distance information between the blank sub-region and the visual center region, and determine the peripheral visual interference coefficient of the blank sub-region based on the visual layout feature information;

[0199] The distance information and the surrounding visual interference coefficient are weighted and summed to obtain the comprehensive score of the dynamic information display of the blank sub-region;

[0200] The blank sub-area with the highest overall score for the dynamic information display is determined as the target display location.

[0201] In this embodiment, a feature acquisition module is used to acquire the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area; an information selection module is used to perform multimodal fusion analysis on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain a user intent classification result, and select target dynamic information from a preset dynamic information database based on the user intent classification result; a position determination module is used to perform visual saliency analysis on the visual layout feature information to obtain a visual center area, and predict the target display position based on the visual center area and the interaction behavior sequence, and dynamically adjust the document object model according to the target display position to generate a dynamic information container, and render and display the target dynamic information in the dynamic information container. The above-mentioned dynamic information position determination device, by fusing and analyzing user interaction behavior and target interaction area features, achieves personalized matching and dynamic optimization of dynamic information display content and display position, thereby improving the dynamic information reach rate.

[0202] The dynamic information location determination device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0203] The dynamic information location determination device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0204] The dynamic information location determination device provided in this application embodiment can realize the various processes implemented in the above embodiments. To avoid repetition, it will not be described again here.

[0205] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a program or instructions stored in the memory 602 and executable on the processor 601. When the program or instructions are executed by the processor 601, they implement the various processes of the above-described dynamic information location determination method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0206] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0207] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described dynamic information location determination method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0208] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0209] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of computer software information. This computer software information is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0211] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0212] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for determining dynamic information location, characterized in that, The method includes: Obtain the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area; Multimodal fusion analysis is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain user intent classification results, and target dynamic information is selected from a preset dynamic information database based on the user intent classification results; Visual saliency analysis is performed on the visual layout feature information to obtain the visual center region. The target interaction area is divided into multiple display areas, and the visual center region and the interaction positions corresponding to each interaction behavior in the interaction behavior sequence are mapped to the corresponding display areas to obtain a display area sequence. The assignment weights corresponding to the transfer path information are determined based on the interaction behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence. For each display area, the weighted number of transfers from the display area to other display areas is calculated based on the transfer path information and the assignment weights. The state transition probability of the display area to other display areas is determined based on the weighted number of transfers from the display area to other display areas to obtain a state transition probability matrix. The target display position is predicted based on the state transition probability matrix, which is used to dynamically adjust the document object model according to the target display position and generate a dynamic information container, in which the target dynamic information is rendered and displayed.

2. The dynamic information location determination method according to claim 1, characterized in that, The multimodal fusion analysis of the document object model, the visual layout feature information, and the interaction behavior sequence to obtain the user intent classification result includes: Feature extraction is performed on the document object model, the visual layout feature information, and the interaction behavior sequence to obtain document structure features, visual layout features, and interaction behavior features; Semantic alignment is performed on the document structure features, the visual layout features, and the interactive behavior features to obtain a cross-modal unified semantic feature vector; The cross-modal unified semantic feature vector is input into a pre-trained intent classification model to obtain the user intent classification result output by the intent classification model; wherein, the intent classification model is jointly optimized and trained based on intent classification loss, cross-modal feature alignment loss and dynamic information transformation association loss.

3. The dynamic information location determination method according to claim 2, characterized in that, The step of semantically aligning the document structure features, visual layout features, and interactive behavior features to obtain a cross-modal unified semantic feature vector includes: The document structure features, the visual layout features, and the interactive behavior features are respectively mapped to a shared latent semantic space; In the shared latent semantic space, the attention weights of the visual layout features and the interactive behavior features relative to the document structure features are calculated; Based on the attention weights, the visual layout features and the interactive behavior features are enhanced to obtain enhanced visual features and enhanced behavior features. The document structure features, the enhanced visual features, and the enhanced behavioral features are concatenated, and the concatenation result is reduced in dimensionality and integrated to obtain a cross-modal unified semantic feature vector.

4. The dynamic information location determination method according to claim 1, characterized in that, The prediction of the target display location based on the state transition probability matrix includes: The display area corresponding to the current interactive behavior is obtained as the first initial display area, and the display area corresponding to the visual center area is obtained as the second initial display area. Based on the state transition probability matrix, the cumulative arrival probability of each display area within a preset number of transitions is calculated, starting from the first initial display area and the second initial display area. The cumulative arrival probability of the target for each display area is obtained by weighted summation of the two cumulative arrival probabilities of each display area. The target display location is determined based on the display area with the highest cumulative arrival probability of the target.

5. The dynamic information location determination method according to claim 4, characterized in that, Determining the target display location based on the display area with the highest cumulative arrival probability of the target includes: Based on the visual layout feature information and the preset dynamic information container size, at least one blank sub-area is selected within the display area where the cumulative arrival probability of the target is highest. Calculate the distance information between the blank sub-region and the visual center region, and determine the peripheral visual interference coefficient of the blank sub-region based on the visual layout feature information; The distance information and the surrounding visual interference coefficient are weighted and summed to obtain the comprehensive score of the dynamic information display of the blank sub-region; The blank sub-area with the highest overall score for the dynamic information display is determined as the target display location.

6. A dynamic information location determination device, characterized in that, The device includes: The feature acquisition module is used to acquire the document object model and visual layout feature information of the target interaction area, and record the user's interaction behavior sequence on the target interaction area; The information selection module is used to perform multimodal fusion analysis on the document object model, the visual layout feature information and the interaction behavior sequence to obtain the user intent classification result, and select target dynamic information from the preset dynamic information database based on the user intent classification result; The location determination module is used to perform visual saliency analysis on the visual layout feature information to obtain a visual center region, divide the target interaction area into multiple display areas, and map the visual center region and the interaction positions corresponding to each interaction behavior in the interaction behavior sequence to the corresponding display areas to obtain a display area sequence; determine the assignment weight corresponding to the transfer path information based on the interaction behavior attribute information corresponding to the transfer path information between adjacent display areas in the display area sequence; for each display area, calculate the weighted number of transfers from the display area to other display areas based on the transfer path information and the assignment weight; determine the state transition probability of the display area to other display areas based on the weighted number of transfers from the display area to other display areas to obtain a state transition probability matrix; predict the target display position based on the state transition probability matrix, and dynamically adjust the document object model according to the target display position to generate a dynamic information container, and render and display the target dynamic information in the dynamic information container.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the dynamic information location determination method as described in any one of claims 1-5.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the dynamic information location determination method as described in any one of claims 1-5.

Images