A method, apparatus, and computer device for online image display

Abstract

This application provides a method, apparatus, and computer device for online image display, relating to the field of image processing technology. The method acquires exhibit images and generates their feature parameters, while simultaneously collecting semantic tags, real-time system status parameters, and user operation intent data. An exhibit image analysis model generates prediction results including viewing value ratings, predicted visitor dwell time, and layout type recommendations. Subsequently, combining real-time status parameters and operation intent data, a layout strategy selection model selects the optimal strategy ID. Based on the ID, a preset strategy script is invoked and executed, ultimately driving a multi-screen splicing display system to complete the adaptive display of the images. This method achieves intelligent generation and optimization of display strategies, improving display effects and viewing experience.

Description

Technical Field

[0001] This application relates to the field of image display technology, and in particular to a method, apparatus, and computer device for online image display. Background Technology

[0002] With the rapid development of multimedia exhibition technology, multi-screen splicing display systems have been widely used in exhibition venues such as museums and science and technology museums. Traditional display systems typically use fixed layouts or manual presets to display images, and their display strategies lack adaptability to the characteristics of exhibits, real-time system status, and audience needs. Existing technologies mostly rely on hardware splicing processors to achieve basic display functions. Although they can perform multi-window splicing and roaming operations, the formulation of display strategies depends entirely on human experience, making it difficult to achieve intelligent decision-making.

[0003] The existing technology has the following main problems: First, the display effect is poor. The system cannot automatically optimize the layout scheme according to the characteristics of the exhibits, resulting in important visual elements not being highlighted. Second, the viewing effect is poor. Due to the lack of comprehensive consideration of audience behavior prediction and system status, there are often phenomena such as screen desynchronization, display delay, and unreasonable layout, which seriously affect the viewing experience. Summary of the Invention

[0004] The technical problem this application aims to solve is to provide a method, apparatus, and computer device for online image display, and to improve the display effect and viewing experience. The specific solution is as follows:

[0005] A method for displaying images online, applied to a multi-screen splicing display system, the method comprising:

[0006] In response to the image display command, retrieve the images of the exhibits to be displayed;

[0007] Based on the exhibit images, generate image feature parameters;

[0008] The semantic tags of the exhibit images, the real-time status parameters of the multi-screen splicing display system, and the operation intent data from the user terminal are obtained; the real-time status parameters include screen unit synchronization error and decoding rendering link load rate.

[0009] The exhibit images, semantic tags, and image feature parameters are input into a pre-trained exhibit image analysis model to obtain image display prediction results. The image display prediction results include the predicted viewing value score, the predicted visitor dwell time, and the probability distribution of the recommended layout type.

[0010] The image display prediction result, the real-time state parameters, and the operation intention data are input into a pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model.

[0011] Based on the strategy ID, the corresponding preset layout strategy script is called from the preset strategy library;

[0012] The preset layout strategy script is executed to generate control instructions and drive the multi-screen splicing display system to complete the online display of the exhibit images.

[0013] Optionally, in the above method, generating image feature parameters based on the exhibit image includes:

[0014] Calculate the information density gradient of the exhibit image, which is used to characterize the rate of change of information complexity in different regions of the exhibit image;

[0015] The visual saliency intensity distribution map of the exhibit images is extracted using a visual saliency model, and the average visual saliency intensity of the exhibit images is calculated.

[0016] The information density gradient and the average visual saliency intensity are used to form the image feature parameters.

[0017] Optionally, the training process of the exhibit image analysis model includes the following steps:

[0018] Obtain a first training dataset and an initial analysis model to be trained; the first training dataset includes multiple first training samples and the ground truth label corresponding to each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth label includes historical viewing value ratings, historical actual visitor dwell time, and the best layout type annotated by historical operators.

[0019] Select the first target training sample for current training from the first training dataset;

[0020] The first target training sample is input into the initial analysis model to obtain the output result of the initial analysis model;

[0021] Calculate the loss function value between the output result and the true value label based on the preset multi-task loss function;

[0022] The parameters of the initial analysis model are updated based on the loss function value;

[0023] If the model after updating the model parameters does not meet the first training completion condition, return to the step of selecting the first target training sample for current training from the first training dataset;

[0024] When the updated model parameters meet the first training completion condition, the model that meets the condition is identified as the trained exhibit image analysis model.

[0025] Optionally, the training process of the layout strategy selection model in the above method includes:

[0026] Obtain a second training dataset and an initial layout strategy selection model to be trained; the second training dataset includes multiple second training samples and a sample strategy ID label corresponding to each second training sample; each second training sample includes historical image display prediction results, historical real-time state parameters, and historical operation intent data;

[0027] Select the second target training sample for the current training from the second training dataset;

[0028] The second target training sample is input into the initial layout strategy selection model to obtain the prediction strategy ID;

[0029] Calculate the loss function value between the prediction strategy ID and the sample strategy ID label based on the preset loss function;

[0030] The parameters of the initial layout strategy selection model are updated based on the loss function value;

[0031] If the model does not meet the second training completion condition after updating the model parameters, return to the step of selecting the second target training sample for current training from the second training dataset;

[0032] When the updated model parameters meet the second training completion condition, the model that meets the condition is identified as the trained layout strategy selection model.

[0033] Optionally, after executing the preset layout strategy script, the above method further includes:

[0034] Monitor the actual duration of visitor stay;

[0035] A model reward signal is generated based on the degree of agreement between the actual audience dwell time and the predicted audience dwell time.

[0036] The exhibit image, the semantic tag, the image feature parameters, the image display prediction result, the real-time status parameters, the operation intention data, the strategy ID, and the model reward signal are stored as a decision record in the historical database.

[0037] The historical database decision records are used at preset time intervals to incrementally train the exhibit image analysis model and / or the layout strategy selection model.

[0038] Optionally, the semantic tags in the above method may include at least one of the following: exhibit type, historical period, relevance to the exhibition theme, and importance level.

[0039] An online image display device, applied to a multi-screen splicing display system, the device comprising:

[0040] The first acquisition unit is used to acquire the exhibit image to be displayed in response to the image display instruction;

[0041] The generation unit is used to generate image feature parameters based on the exhibit images;

[0042] The second acquisition unit is used to acquire the semantic tags of the exhibit images, the real-time status parameters of the multi-screen splicing display system, and the operation intent data from the user terminal; the real-time status parameters include screen unit synchronization error and decoding rendering link load rate;

[0043] The first execution unit is used to input the exhibit image, the semantic tag, and the image feature parameters into a pre-trained exhibit image analysis model to obtain the image display prediction result; the image display prediction result includes the predicted viewing value score, the predicted audience dwell time, and the probability distribution of the recommended layout type;

[0044] The second execution unit is used to input the image display prediction result, the real-time state parameters, and the operation intention data into a pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model.

[0045] The calling unit is used to call the corresponding preset layout strategy script from the preset strategy library according to the strategy ID;

[0046] The third execution unit is used to execute the preset layout strategy script to generate control instructions and drive the multi-screen splicing display system to complete the online display of the exhibit images.

[0047] Optionally, in the aforementioned apparatus, the generating unit includes:

[0048] The first calculation subunit is used to calculate the information density gradient of the exhibit image, wherein the information density gradient is used to characterize the rate of change of information complexity in different regions of the exhibit image;

[0049] The second calculation subunit is used to extract the visual saliency intensity distribution map of the exhibit image through the visual saliency model, and to calculate the average visual saliency intensity of the exhibit image.

[0050] The information density gradient and the average visual saliency intensity are used to form the image feature parameters.

[0051] Optionally, in the aforementioned apparatus, the first execution unit includes:

[0052] The first acquisition subunit is used to acquire the first training dataset and the initial analysis model to be trained; the first training dataset includes multiple first training samples and the ground truth label corresponding to each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth label includes historical viewing value rating, historical visitor actual dwell time, and the best layout type marked by historical operators.

[0053] Select a sub-unit to select the first target training sample for current training from the first training dataset;

[0054] The first execution subunit is used to input the first target training sample into the initial analysis model and obtain the output result of the initial analysis model;

[0055] The third calculation subunit is used to calculate the loss function value between the output result and the true value label according to the preset multi-task loss function;

[0056] An update subunit is used to update the parameters of the initial analysis model based on the loss function value;

[0057] The second execution subunit is used to return to trigger the selection subunit to execute the step of selecting the first target training sample for current training from the first training dataset when the model after updating the model parameters does not meet the first training completion condition.

[0058] The third execution subunit is used to determine the model that meets the first training completion condition as the trained exhibit image analysis model when the model after updating the model parameters meets the first training completion condition.

[0059] The aforementioned apparatus may optionally further include:

[0060] The monitoring unit is used to monitor the actual duration of visitor stays.

[0061] The fourth execution unit is used to generate a model reward signal based on the degree of agreement between the actual audience dwell time and the predicted audience dwell time;

[0062] The storage unit is used to store the exhibit image, the semantic tag, the image feature parameters, the image display prediction result, the real-time status parameters, the operation intention data, the strategy ID, and the model reward signal as a decision record in the historical database.

[0063] The training unit is used to incrementally train the exhibit image analysis model and / or the layout strategy selection model using decision records in the historical database at preset time intervals.

[0064] A computer device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described above for displaying images online.

[0065] Based on the above-described embodiments of this application, an online image display method, apparatus, and computer device are provided, wherein, in response to an image display instruction, an image of an exhibit to be displayed is acquired; image feature parameters are generated based on the exhibit image; semantic tags of the exhibit image, real-time status parameters of the multi-screen splicing display system, and operation intent data from the user terminal are acquired; the real-time status parameters include screen unit synchronization error and decoding rendering link load rate; the exhibit image, the semantic tags, and the image feature parameters are input into a pre-trained exhibit image analysis model to obtain an image display prediction result; the image... The image display prediction results include the predicted viewing value score, the predicted audience dwell time, and the probability distribution of the recommended layout type. The image display prediction results, the real-time state parameters, and the operation intent data are input into a pre-trained layout strategy selection model to obtain a strategy ID output by the model. The layout strategy selection model is a gradient boosting decision tree model. Based on the strategy ID, a corresponding preset layout strategy script is called from a preset strategy library. The preset layout strategy script is executed to generate control commands and drive the multi-screen splicing display system to complete the online display of the exhibit images. This method achieves intelligent generation and optimization of display strategies, improving display effects and viewing experience. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0067] Figure 1A flowchart illustrating a method for online image display provided in this application;

[0068] Figure 2 A flowchart of a process for generating image feature parameters is provided in this application;

[0069] Figure 3 A schematic diagram of the structure of an online image display device provided in this application;

[0070] Figure 4 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0072] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0073] This invention provides a method for online image display, applied to a control device in a multi-screen splicing display system. The flowchart of the method is shown below. Figure 1 As shown, it specifically includes:

[0074] S101: In response to the image display command, obtain the image of the exhibit to be displayed.

[0075] In this embodiment, the triggering mechanism for image display instructions includes, but is not limited to, the following: automatic display instructions triggered at preset intervals, manual triggering instructions sent by the user through the control terminal, or system triggering instructions generated in response to exhibit update events. The control terminal includes one or more of the following: a computer, a mobile device, a dedicated controller, and an IoT control node.

[0076] Optionally, the exhibit images are digitized images of the exhibits, sourced from sources including: high-resolution scanned images of cultural relics, digital copies of artworks, visualizations of scientific exhibits, digitized images of historical archives, and other image materials of exhibition value. The image resolution of the exhibit images is adapted to the physical resolution characteristics of the display units in the multi-screen splicing display system.

[0077] Specifically, the process of acquiring exhibit images is implemented through the application programming interface (API) of the image management system. This API supports RESTful architecture calls based on HTTP and HTTPS protocols or dedicated communication protocols based on TCP / IP. For exhibit images stored locally, direct access to the file system is used; for image resources stored in the cloud, download and cache are performed using the SDK provided by the cloud storage service after a security authentication mechanism.

[0078] In this embodiment, the metadata of the exhibit image includes, but is not limited to: image size, color space, creation time, modification time, copyright information, and digital watermark features. During the acquisition process, the integrity and authorization status of the image are simultaneously verified, and encrypted images are decrypted using corresponding decryption algorithms to ensure that the image can be decoded and displayed correctly.

[0079] Optionally, for large-scale exhibit image libraries, a preloading mechanism can be used to cache designated exhibit images to local storage devices in advance according to the exhibition plan, reducing network latency during real-time retrieval. For ultra-high resolution images, a chunked transmission and progressive loading technique is used to prioritize loading and displaying key areas of the image.

[0080] S102: Generate image feature parameters based on exhibit images.

[0081] In this embodiment, the image feature parameters refer to the set of quantified features obtained by calculating the visual attributes of the exhibit image, including at least the information density gradient and the visual salience intensity.

[0082] Optionally, the information density gradient is used to characterize the rate of change of information complexity in local regions of the image, and is obtained by calculating the entropy difference between adjacent pixel blocks in the image; the visual saliency intensity is obtained by extracting the distribution features of salient regions through a visual saliency model and then calculating its average intensity value.

[0083] In some embodiments, the image feature parameters may further include one or more of color distribution features, texture complexity indices, and spatial frequency features. Color distribution features are obtained by calculating the histogram distribution of the image in the color space; texture complexity indices are calculated using the local binary mode algorithm or the gray-level co-occurrence matrix method; spatial frequency features are extracted after analyzing the image's frequency domain characteristics using fast Fourier transform.

[0084] In this embodiment, the image feature parameter generation process is implemented through a feature calculation engine, which includes an information entropy calculation module, a gradient analysis module, and a visual saliency processing module. Each module adopts a parallel computing architecture, using GPU acceleration to improve feature extraction efficiency and ensure the real-time generation of the required image feature parameters.

[0085] Optionally, the generated image feature parameters are output in a structured data format, including JSON or Protocol Buffers, containing fields such as feature name, numerical vector, and confidence index.

[0086] S103: Obtain semantic tags for exhibit images, real-time status parameters of the multi-screen splicing display system, and operation intent data from user terminals; real-time status parameters include screen unit synchronization error and decoding rendering link load rate.

[0087] In this embodiment, semantic tags refer to structured descriptive information obtained after content analysis of exhibit images using image recognition technology. Specifically, they include at least one of exhibit type classification, historical period identification, thematic relevance, and importance grading. Semantic tags are generated through a pre-trained image classification model. This model employs a deep convolutional neural network architecture and is fine-tuned on a professional exhibit dataset using transfer learning. This model accurately identifies the content features of various exhibits and outputs corresponding semantic descriptive tags.

[0088] Optionally, the real-time status parameters of the multi-screen splicing display system include screen unit synchronization error and decoding rendering link load rate. The screen unit synchronization error is obtained by measuring the frame buffer state differences between each display unit. Specifically, a high-precision timestamp comparison algorithm is used to calculate the image output time difference between adjacent display units, and the screen unit synchronization error can be quantified in microseconds. The decoding rendering link load rate can be obtained by monitoring the resource usage of the video processor, including indicators such as GPU decoding utilization, video memory usage, and PCIe bus bandwidth utilization, and a weighted calculation is used to obtain the comprehensive load rate value.

[0089] Optionally, operation intent data refers to structured control parameters obtained after parsing the operation commands issued by the user through the control terminal, including but not limited to layout mode selection commands, window arrangement preferences, display effect parameters, and priority settings. These structured control parameters are generated by parsing user interaction events and are structured and encapsulated in JSON or XML format, containing fields such as operation type, parameter list, and timestamp.

[0090] Specifically, semantic tags can be obtained by calling the interface of the image analysis service; real-time status parameters can be collected in real time through the status monitoring interface of the display control system, with a sampling frequency of not less than 1Hz; operation intention data can be received from the user control terminal through the message middleware, and after parsing and verification, it is converted into a structured format that can be processed internally by the system.

[0091] In some embodiments, real-time status parameters may also include one or more of temperature sensor readings, fan speed monitoring data, and power status indicators. The acquisition of operation intent data supports multiple transmission protocols, including WebSocket real-time communication protocol, MQTT IoT protocol, and custom binary protocol, ensuring reliable acquisition of user operation data in different network environments.

[0092] S104: Input the exhibit images, semantic tags, and image feature parameters into the pre-trained exhibit image analysis model to obtain the image display prediction results; the image display prediction results include the predicted viewing value score, the predicted visitor dwell time, and the probability distribution of the recommended layout type.

[0093] In this embodiment, the exhibit image analysis model is a prediction model based on a multi-task deep learning architecture, employing a network structure that combines shared low-level feature extraction with independent task output branches. The shared feature extraction layer is implemented using a deep convolutional neural network to extract common features from the input data; the independent task output branches include three fully connected sub-networks, corresponding to the three prediction tasks of viewing value rating prediction, visitor dwell time prediction, and layout type classification, respectively.

[0094] Specifically, the viewing value score prediction is achieved using a regression algorithm, with an output range of 0-100 points, reflecting the artistic value, visual appeal, and content importance of the exhibit images. The prediction of visitor dwell time is achieved using a time series regression algorithm, outputting an estimated viewing time in seconds. This prediction is based on the correlation analysis between historical visitor behavior data and image content features. The layout type probability distribution can be achieved using a softmax classifier, outputting the selection probability distribution of various preset layout types, including full-screen display, picture-in-picture, multi-window tiling, and magnification of key areas.

[0095] In this embodiment, the training process of the exhibit image analysis model adopts a multi-task joint learning approach, using a training dataset that includes exhibit images, semantic labels, image feature parameters, and corresponding ground truth annotations. The loss function is a weighted sum of the losses from each task. Model optimization employs an adaptive moment estimation optimizer, updating network parameters through a backpropagation algorithm.

[0096] Optionally, the model supports an online learning mechanism, allowing for dynamic adjustment of model parameters based on feedback data from actual display results. The model inference process employs batch processing and supports GPU-accelerated computation to ensure real-time performance. Image display prediction results are output in a structured data format, including numerical results for each prediction and corresponding confidence indices, providing data support for subsequent layout strategy selection.

[0097] Optionally, the input data for the exhibit image analysis model needs to be preprocessed, including image size normalization, feature value standardization, and label encoding conversion. Post-processing of the output results includes numerical range constraints, probability distribution normalization, and outlier filtering to ensure the rationality and usability of the prediction results.

[0098] S105: Input the image display prediction results, real-time state parameters, and operation intention data into the pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model.

[0099] In this embodiment, the placement strategy selection model is constructed using the gradient boosting decision tree algorithm. It achieves high-precision strategy selection by integrating multiple weak classifiers (decision trees) to form a strong classifier. The model comprises three core components: a base learner generation module, a gradient boosting optimization module, and a policy output module. The base learner uses a regression decision tree structure, iteratively reducing prediction errors. The gradient boosting optimization module employs a negative gradient direction optimization strategy, continuously improving the model's predictive ability by minimizing the exponential loss function. The policy output module uses a softmax function to convert the final prediction result into a probability distribution of policy IDs, taking the policy ID corresponding to the highest probability value as the final output.

[0100] Specifically, the layout strategy selection model employs supervised learning during training. The training dataset includes historical image display prediction results, historical real-time state parameters, historical operation intent data, and corresponding optimal strategy ID labels. During model training, the input features are first standardized and preprocessed, and then ranked by feature importance. A sequence of decision trees is iteratively constructed using a forward stepwise algorithm, with each decision tree learning the residuals of all previous trees. The model employs an early stopping mechanism to prevent overfitting, and cross-validation is used to determine the optimal number of iterations.

[0101] In this embodiment, the strategy ID is a unique identifier in the preset strategy library, using integer encoding. Each ID corresponds to a specific display layout strategy. The strategy ID output by the strategy selection model is converted into executable layout instructions by the strategy parser, including window layout configuration, display parameter settings, and animation effect parameters.

[0102] Optionally, the layout strategy selection model supports an online update mechanism, allowing for dynamic adjustment of model parameters based on feedback data from actual display effects. The model employs batch processing and supports multi-threaded parallel computation to ensure real-time response. The model output includes a strategy ID and its corresponding confidence score; a manual intervention mechanism is activated when the confidence score falls below a preset threshold.

[0103] The input data for the layout strategy selection model requires feature engineering, including numerical feature standardization, one-hot encoding of categorical features, and sliding window processing of temporal features. Post-processing of the output results includes confidence verification, strategy ID validity validation, and outlier filtering to ensure the reliability and feasibility of the output strategy.

[0104] S106: Based on the strategy ID, call the corresponding preset layout strategy script from the preset strategy library.

[0105] In this embodiment, the policy ID is a unique coded identifier that identifies a specific layout policy. It adopts an integer encoding system, and each policy ID is mapped one-to-one with a specific policy script in the preset policy library. The preset policy library is a pre-configured policy storage system, implemented using a relational database or key-value pair storage structure, where the policy ID serves as the primary key or key name, and the corresponding policy script is stored as the value.

[0106] Specifically, the preset layout strategy scripts are structured configuration files written in JSON, XML, or YAML format, containing a complete sequence of layout control instructions. Each script defines the following parameters in detail: the number of windows and the position coordinates and size specifications of each window, the display hierarchy, the type and duration of transition animations, transparency settings, border styles, and special effect parameters. For multi-screen splicing environments, the scripts also include cross-screen display configuration, synchronization parameter settings, and color consistency calibration instructions.

[0107] The invocation process is implemented through the policy management engine. After receiving the policy ID as input, the engine first verifies the validity and access permissions of the ID, and then retrieves the corresponding policy script content from the preset policy library through a database query interface or key-value retrieval interface. After the retrieved script undergoes syntax parsing and semantic verification, it is converted into an internally executable intermediate representation format.

[0108] Optionally, the default policy library supports a version management mechanism, allowing the storage of multiple versions of policy scripts simultaneously, and automatically selecting the appropriate version based on system environment parameters. The policy library also provides a remote update interface, supporting dynamic updates and hot reloading of policy scripts via network protocols, with effects taking effect without requiring a system restart.

[0109] In some embodiments, an automatic fix or the latest valid version is activated when a policy script has a syntax error; and parameter standardization is automatically performed when policy parameters exceed the system's capabilities.

[0110] Optionally, the final obtained preset layout strategy script is converted into a sequence of control instructions recognizable by the underlying display control system through a strategy parser, providing accurate and complete operational guidance for subsequent execution phases. All calls are logged in detail, including call time, strategy ID, execution result, and other information, for subsequent auditing and performance analysis.

[0111] S107: Execute the preset layout strategy script to generate control instructions and drive the multi-screen splicing display system to complete the online display of exhibit images.

[0112] In this embodiment, the execution process is implemented through a script execution engine, which includes three core components: a script parsing module, an instruction generation module, and a system driver module. The script parsing module uses a recursive descent parsing algorithm to perform syntax analysis and semantic checks on the preset layout strategy script, converting the script content into an intermediate representation. The instruction generation module generates a sequence of device-level control instructions based on the intermediate representation. The system driver module sends the control instructions to the multi-screen splicing display system through the display control interface.

[0113] Specifically, control instructions include, but are not limited to, the following types: window management instructions, display hierarchy adjustment instructions, special effects control instructions, synchronization control instructions, and resource management instructions. Control instructions are in binary encoding format, containing opcode, operand, and checksum fields, and are transmitted to the display device via a high-speed serial bus or network protocol. Window management instructions are used to instruct the creation, destruction, movement, and scaling of at least one window; display hierarchy adjustment instructions are used to instruct at least one of placing on top, placing at the bottom, and layer swapping; special effects control instructions are used to control transition animations such as fade-in, fly-in, and fade-out; synchronization control instructions are used to perform frame synchronization, vertical synchronization, and clock synchronization; and resource management instructions are used to perform memory allocation, buffer management, and the establishment and release of rendering links.

[0114] In this embodiment, the multi-screen splicing display system includes a control device and multiple display units. The driving process first initializes each display unit through the device driver interface, configuring the display mode and synchronization parameters; then, it sequentially executes window layout operations according to the control command sequence, including calculating the absolute position of each window in the multi-screen coordinate system, handling edge blending and color correction when displaying across screens, and configuring the input source and display parameters of each display channel; finally, it triggers all display units to update the screen simultaneously through a synchronization control signal, ensuring the synchronization and consistency of the screen display.

[0115] Optionally, the command execution timing and parameters can be dynamically adjusted by monitoring the response status and actual display effect of each display unit; when a display abnormality or equipment failure is detected, a fault-tolerant processing mechanism is automatically activated, including command retransmission, backup channel switching, or downgraded display mode switching.

[0116] In this embodiment, the online display of exhibit images supports multiple display modes, including but not limited to: full-screen single-view display mode, picture-in-picture display mode, multi-window side-by-side display mode, focused area magnified display mode, and dynamic roaming display mode. During the display process, system performance indicators, including frame rate, latency, and resource utilization, are monitored in real time to ensure the smoothness and stability of the display effect.

[0117] In some embodiments, an operation log is generated after execution, containing information such as execution time, execution result, and system state changes, for subsequent auditing and performance analysis. All control instructions are executed using a transaction processing mechanism to ensure the atomicity and consistency of operations, and to allow rollback to the previous stable state in the event of an exception.

[0118] By applying the method provided in the embodiments of this application, intelligent generation and optimization of display strategies are realized, thereby improving the display effect and viewing experience.

[0119] In one embodiment provided in this application, based on the above-described solution, optionally, the process of generating image feature parameters based on exhibit images is as follows: Figure 2 As shown, it includes:

[0120] S201: Calculate the information density gradient of the exhibit image. The information density gradient is used to characterize the rate of change of information complexity in different regions of the exhibit image.

[0121] In this embodiment, the calculation process of the information density gradient includes: first, dividing the exhibit image into several pixel blocks, and calculating the information entropy value of each pixel block using a sliding window method; then, calculating the gradient magnitude of the entropy value of each pixel block using the Sobel operator or other edge detection algorithms, and finally generating a gradient distribution map representing the rate of change of information complexity, the gradient distribution map including the information density gradient of the exhibit image.

[0122] S202: Extract the visual saliency intensity distribution map of the exhibit images using the visual saliency model, and calculate the average visual saliency intensity of the exhibit images.

[0123] Optionally, the calculation process for the average visual saliency intensity includes: using a deep learning-based visual saliency model, such as a saliency detection network with a U-Net architecture, to process the exhibit images and generate a visual saliency heatmap; after normalizing the heatmap, calculating the mean intensity of all pixels to obtain the average visual saliency intensity.

[0124] S203: The information density gradient and the average visual saliency intensity are combined to form the image feature parameters.

[0125] In one embodiment provided in this application, based on the above-described solution, optionally, the training process of the exhibit image analysis model includes:

[0126] Obtain the first training dataset and the initial analysis model to be trained; the first training dataset includes multiple first training samples and the corresponding ground truth labels for each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth labels include historical viewing value scores, historical actual visitor dwell time, and the best layout type annotated by historical operators.

[0127] Select the first target training sample for current training from the first training dataset;

[0128] Input the first target training sample into the initial analysis model to obtain the output result of the initial analysis model;

[0129] Based on the preset multi-task loss function, calculate the loss function value between the output result and the true value label;

[0130] Update the parameters of the initial analysis model based on the loss function value;

[0131] If the model does not meet the first training completion condition after updating the model parameters, return to the step of selecting the first target training sample for current training from the first training dataset;

[0132] When the updated model parameters meet the first training completion condition, the model that meets the condition is identified as the trained exhibit image analysis model.

[0133] In this embodiment, the first training dataset consists of multiple first training samples, each containing three input feature components: historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images. The corresponding ground truth labels include three supervisory signals: historical viewing value rating, historical visitor actual dwell time, and the optimal layout type annotated by historical operators. The initial analysis model adopts a multi-task deep learning architecture, including a shared feature extraction layer and three independent task-specific output layers.

[0134] Optionally, the loss function consists of three parts: the mean squared error loss function is used for viewing value rating prediction, the Huber loss function is used for audience dwell time prediction, and the cross-entropy loss function is used for layout type prediction. The total loss value is the weighted sum of the losses of each task, and the weight coefficients are dynamically adjusted according to the importance of each task. Based on the calculated loss function value, the parameters of the initial analysis model are updated using the backpropagation algorithm. Parameter optimization uses an adaptive moment estimation optimizer, and the learning rate adopts an exponential decay strategy. After the update is completed, the model is evaluated to see if it meets the first training completion condition, which may include: the loss function value is lower than a preset threshold, the number of training epochs reaches its maximum value, or the accuracy of the validation set has not improved for several consecutive epochs.

[0135] In one embodiment provided in this application, based on the above-described scheme, optionally, the training process of the layout strategy selection model includes:

[0136] Obtain the second training dataset and the initial layout strategy selection model to be trained; the second training dataset includes multiple second training samples and the sample strategy ID label corresponding to each second training sample; each second training sample includes historical image display prediction results, historical real-time state parameters, and historical operation intent data;

[0137] Select the second target training sample for the current training from the second training dataset;

[0138] The training samples of the second target are input into the initial layout strategy selection model to obtain the prediction strategy ID;

[0139] Calculate the loss function value between the prediction strategy ID and the sample strategy ID label based on the preset loss function;

[0140] Update the parameters of the initial layout strategy selection model based on the loss function value;

[0141] If the updated model parameters do not meet the second training completion condition, return to the step of selecting the second target training sample for current training from the second training dataset;

[0142] When the updated model parameters meet the second training completion condition, the model that meets the condition is identified as the trained layout strategy selection model.

[0143] In this embodiment, the second training dataset consists of multiple second training samples. Each second training sample contains three input feature components: historical image display prediction results, historical real-time state parameters, and historical operation intent data. The corresponding sample policy ID label is a validated best policy identifier, which consists of policy IDs that have been actually adopted and have shown good display effects, as recorded in the system logs. The initial layout policy selection model adopts a gradient boosting decision tree architecture, which includes multiple regression decision trees and corresponding weight parameters.

[0144] The loss function uses a multi-class cross-entropy loss function, and its mathematical expression is as follows:

[0145]

[0146] in, One-hot encoding of the sample policy ID. This represents the probability distribution predicted by the model. For class imbalance, a weighted cross-entropy loss function can be used for adjustment. Based on the calculated loss function value, the parameters of the initial layout strategy selection model are updated using a gradient boosting algorithm. Specifically, the gradient of the loss function with respect to the model parameters is calculated, and the structure and weight parameters of the decision tree are iteratively optimized using gradient descent.

[0147] During each iteration, the newly added decision tree learns from the prediction residuals of all previous trees, gradually improving the model's accuracy. After the update is complete, the model is evaluated to determine whether it meets the second training completion condition, which includes: the loss function value converges to a stable interval, the validation set accuracy reaches a preset threshold, or the number of decision trees reaches the maximum limit.

[0148] In one embodiment provided in this application, based on the above solution, optionally, after executing the preset layout strategy script, the method further includes:

[0149] Monitor the actual duration of visitor stay;

[0150] A model reward signal is generated based on the degree of agreement between the actual audience dwell time and the predicted audience dwell time.

[0151] The exhibit images, semantic tags, image feature parameters, image display prediction results, real-time status parameters, operation intention data, strategy ID, and model reward signals are stored as a decision record in the historical database.

[0152] The exhibit image analysis model and / or layout strategy selection model are incrementally trained using decision records from the historical database at preset time intervals.

[0153] In this embodiment, an infrared sensor array or computer vision monitoring system deployed in the exhibition area collects data on the actual dwell time of visitors in front of the exhibits. The monitoring system continuously records the behavior data of each visitor with a time accuracy of seconds. After data denoising and outlier filtering, the arithmetic mean of the dwell time of all visitors is taken as the actual dwell time of visitors.

[0154] Optionally, a model reward signal is generated based on the degree of agreement between the actual and predicted audience dwell times. The degree of agreement is calculated using a relative error algorithm, and the formula is as follows:

[0155]

[0156] The reward signal takes continuous values ​​in the range [0,1], where 1 represents a perfect match and 0 represents a complete deviation. An adaptive threshold mechanism is also set up to trigger a model retraining alert when the match score consistently falls below 0.6.

[0157] In this embodiment, the decision records are stored in a time-series database, with each record containing a timestamp, exhibit identifier, and complete decision context data. The database employs a columnar storage structure to optimize query performance and establishes a composite index to support multi-dimensional data analysis.

[0158] The exhibit image analysis model and / or layout strategy selection model are incrementally trained using decision records from the historical database at preset time intervals. These time intervals can be dynamically adjusted based on system load, typically set to 24 hours or triggered when 1000 new decision records are accumulated. During incremental training, high-quality decision records with reward signals greater than 0.8 are prioritized as training samples, and online learning algorithms are used to progressively update the model parameters.

[0159] Optionally, the incremental training process supports a hot update mechanism, which can update model parameters without affecting the normal operation of the system. A model version management system is also established to retain model parameters from historical versions, allowing for quick rollback to a stable version when the performance of a new model deteriorates. Detailed logs are recorded for all training operations, including training time, data size, and model performance metrics, for performance evaluation and problem identification.

[0160] Through the above mechanism, the system can continuously optimize the quality of the generated display strategies, forming a complete closed-loop learning system from decision execution to effect evaluation to model optimization, and continuously improving the effect of exhibit display and audience experience.

[0161] See Figure 3 This is a schematic diagram of an online image display device provided in an embodiment of this application. The online image display device is applied to the control equipment of a multi-screen splicing display system, and the device includes:

[0162] The first acquisition unit 301 is used to acquire the exhibit image to be displayed in response to the image display instruction;

[0163] The generation unit 302 is used to generate image feature parameters based on the exhibit images;

[0164] The second acquisition unit 303 is used to acquire the semantic tags of the exhibit images, the real-time status parameters of the multi-screen splicing display system, and the operation intent data from the user terminal; the real-time status parameters include screen unit synchronization error and decoding rendering link load rate;

[0165] The first execution unit 304 is used to input the exhibit image, the semantic tag, and the image feature parameters into a pre-trained exhibit image analysis model to obtain an image display prediction result; the image display prediction result includes the predicted viewing value score, the predicted audience dwell time, and the probability distribution of the recommended layout type.

[0166] The second execution unit 305 is used to input the image display prediction result, the real-time state parameters, and the operation intention data into a pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model.

[0167] Calling unit 306 is used to call the corresponding preset layout strategy script from the preset strategy library according to the strategy ID;

[0168] The third execution unit 307 is used to execute the preset layout strategy script to generate control instructions and drive the multi-screen splicing display system to complete the online display of the exhibit images.

[0169] In one embodiment provided in this application, based on the above-described solution, optionally, the generation unit 302 includes:

[0170] The first calculation subunit is used to calculate the information density gradient of the exhibit image, wherein the information density gradient is used to characterize the rate of change of information complexity in different regions of the exhibit image;

[0171] The second calculation subunit is used to extract the visual saliency intensity distribution map of the exhibit image through the visual saliency model, and to calculate the average visual saliency intensity of the exhibit image.

[0172] The information density gradient and the average visual saliency intensity are used to form the image feature parameters.

[0173] In one embodiment provided in this application, based on the above-described solution, optionally, the first execution unit 304 includes:

[0174] The first acquisition subunit is used to acquire the first training dataset and the initial analysis model to be trained; the first training dataset includes multiple first training samples and the ground truth label corresponding to each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth label includes historical viewing value rating, historical visitor actual dwell time, and the best layout type marked by historical operators.

[0175] Select a sub-unit to select the first target training sample for current training from the first training dataset;

[0176] The first execution subunit is used to input the first target training sample into the initial analysis model and obtain the output result of the initial analysis model;

[0177] The third calculation subunit is used to calculate the loss function value between the output result and the true value label according to the preset multi-task loss function;

[0178] An update subunit is used to update the parameters of the initial analysis model based on the loss function value;

[0179] The second execution subunit is used to return to trigger the selection subunit to execute the step of selecting the first target training sample for current training from the first training dataset when the model after updating the model parameters does not meet the first training completion condition.

[0180] The third execution subunit is used to determine the model that meets the first training completion condition as the trained exhibit image analysis model when the model after updating the model parameters meets the first training completion condition.

[0181] In one embodiment provided in this application, based on the above-described solution, optionally, it further includes:

[0182] The monitoring unit is used to monitor the actual duration of visitor stays.

[0183] The fourth execution unit is used to generate a model reward signal based on the degree of agreement between the actual audience dwell time and the predicted audience dwell time;

[0184] The storage unit is used to store the exhibit image, the semantic tag, the image feature parameters, the image display prediction result, the real-time status parameters, the operation intention data, the strategy ID, and the model reward signal as a decision record in the historical database.

[0185] The training unit is used to incrementally train the exhibit image analysis model and / or the layout strategy selection model using decision records in the historical database at preset time intervals.

[0186] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0187] This application also provides a computer device, the structural schematic diagram of which is shown below. Figure 4 As shown, it specifically includes a memory 401 and one or more instructions 402, wherein one or more instructions 402 are stored in the memory 401 and configured to be executed by one or more processors 403 to perform the above-described method of online image display.

[0188] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0189] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0190] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0191] The above provides a detailed description of an online image display method provided by this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for displaying images online, characterized in that, The method, applied to a multi-screen splicing display system, includes: In response to the image display command, retrieve the images of the exhibits to be displayed; Based on the exhibit images, generate image feature parameters; The system acquires semantic tags for the exhibit images, real-time status parameters of the multi-screen splicing display system, and operation intent data from the user terminal. The real-time status parameters include screen unit synchronization error and decoding rendering link load rate. The operation intent data includes layout mode selection instructions, window arrangement preferences, display effect parameters, and priority setting information obtained after parsing the operation instructions issued by the user through the control terminal. The exhibit images, semantic tags, and image feature parameters are input into a pre-trained exhibit image analysis model to obtain image display prediction results. The image display prediction results include the predicted viewing value score, the predicted visitor dwell time, and the probability distribution of the recommended layout type. The image display prediction result, the real-time state parameters, and the operation intention data are input into a pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model. Based on the strategy ID, the corresponding preset layout strategy script is called from the preset strategy library; The preset layout strategy script is executed to generate control instructions and drive the multi-screen splicing display system to complete the online display of the exhibit images.

2. The method according to claim 1, characterized in that, The step of generating image feature parameters based on the exhibit images includes: Calculate the information density gradient of the exhibit image, which is used to characterize the rate of change of information complexity in different regions of the exhibit image; The visual saliency intensity distribution map of the exhibit images is extracted using a visual saliency model, and the average visual saliency intensity of the exhibit images is calculated. The information density gradient and the average visual saliency intensity are used to form the image feature parameters.

3. The method according to claim 1, characterized in that, The training process of the exhibit image analysis model includes: Obtain a first training dataset and an initial analysis model to be trained; the first training dataset includes multiple first training samples and the ground truth label corresponding to each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth label includes historical viewing value ratings, historical actual visitor dwell time, and the best layout type annotated by historical operators. Select the first target training sample for current training from the first training dataset; The first target training sample is input into the initial analysis model to obtain the output result of the initial analysis model; Calculate the loss function value between the output result and the true value label based on the preset multi-task loss function; The parameters of the initial analysis model are updated based on the loss function value; If the model after updating the model parameters does not meet the first training completion condition, return to the step of selecting the first target training sample for current training from the first training dataset; When the updated model parameters meet the first training completion condition, the model that meets the condition is identified as the trained exhibit image analysis model.

4. The method according to claim 1, characterized in that, The training process of the layout strategy selection model includes: Obtain a second training dataset and an initial layout strategy selection model to be trained; the second training dataset includes multiple second training samples and a sample strategy ID label corresponding to each second training sample; each second training sample includes historical image display prediction results, historical real-time state parameters, and historical operation intent data; Select the second target training sample for the current training from the second training dataset; The second target training sample is input into the initial layout strategy selection model to obtain the prediction strategy ID; Calculate the loss function value between the prediction strategy ID and the sample strategy ID label based on the preset loss function; The parameters of the initial layout strategy selection model are updated based on the loss function value; If the model does not meet the second training completion condition after updating the model parameters, return to the step of selecting the second target training sample for current training from the second training dataset; When the updated model parameters meet the second training completion condition, the model that meets the condition is identified as the trained layout strategy selection model.

5. The method according to claim 1, characterized in that, After executing the preset layout strategy script, the process also includes: Monitor the actual duration of visitor stay; A model reward signal is generated based on the degree of agreement between the actual audience dwell time and the predicted audience dwell time. The exhibit image, the semantic tag, the image feature parameters, the image display prediction result, the real-time status parameters, the operation intention data, the strategy ID, and the model reward signal are stored as a decision record in the historical database. The historical database decision records are used at preset time intervals to incrementally train the exhibit image analysis model and / or the layout strategy selection model.

6. The method according to claim 1, characterized in that, The semantic tags include at least one of the following: exhibit type, historical period, relevance to the exhibition theme, and level of importance.

7. A device for displaying images online, characterized in that, The device, used in multi-screen splicing display systems, includes: The first acquisition unit is used to acquire the exhibit image to be displayed in response to the image display instruction; The generation unit is used to generate image feature parameters based on the exhibit images; The second acquisition unit is used to acquire the semantic tags of the exhibit images, the real-time status parameters of the multi-screen splicing display system, and operation intent data from the user terminal; the real-time status parameters include screen unit synchronization error and decoding rendering link load rate; the operation intent data includes: layout mode selection instructions, window arrangement preferences, display effect parameters, and priority setting information obtained after parsing the operation instructions issued by the user through the control terminal. The first execution unit is used to input the exhibit image, the semantic tag, and the image feature parameters into a pre-trained exhibit image analysis model to obtain the image display prediction result; the image display prediction result includes the predicted viewing value score, the predicted audience dwell time, and the probability distribution of the recommended layout type; The second execution unit is used to input the image display prediction result, the real-time state parameters, and the operation intention data into a pre-trained layout strategy selection model to obtain the strategy ID output by the layout strategy selection model; the layout strategy selection model is a gradient boosting decision tree model. The calling unit is used to call the corresponding preset layout strategy script from the preset strategy library according to the strategy ID; The third execution unit is used to execute the preset layout strategy script to generate control instructions and drive the multi-screen splicing display system to complete the online display of the exhibit images.

8. The apparatus according to claim 7, characterized in that, The generation unit includes: The first calculation subunit is used to calculate the information density gradient of the exhibit image, wherein the information density gradient is used to characterize the rate of change of information complexity in different regions of the exhibit image; The second calculation subunit is used to extract the visual saliency intensity distribution map of the exhibit image through the visual saliency model, and to calculate the average visual saliency intensity of the exhibit image. The information density gradient and the average visual saliency intensity are used to form the image feature parameters.

9. The apparatus according to claim 7, characterized in that, The first execution unit includes: The first acquisition subunit is used to acquire the first training dataset and the initial analysis model to be trained; the first training dataset includes multiple first training samples and the ground truth label corresponding to each first training sample; each first training sample includes historical exhibit images, historical semantic labels, and historical image feature parameters calculated based on the historical exhibit images; the ground truth label includes historical viewing value rating, historical visitor actual dwell time, and the best layout type marked by historical operators. Select a sub-unit to select the first target training sample for current training from the first training dataset; The first execution subunit is used to input the first target training sample into the initial analysis model and obtain the output result of the initial analysis model; The third calculation subunit is used to calculate the loss function value between the output result and the true value label according to the preset multi-task loss function; An update subunit is used to update the parameters of the initial analysis model based on the loss function value; The second execution subunit is used to return to trigger the selection subunit to execute the step of selecting the first target training sample for current training from the first training dataset when the model after updating the model parameters does not meet the first training completion condition. The third execution subunit is used to determine the model that meets the first training completion condition as the trained exhibit image analysis model when the model after updating the model parameters meets the first training completion condition.

10. A computer device, characterized in that, It includes a memory and a processor, the memory being used to store instructions and the processor being used to execute the method as described in any one of claims 1-6.

Images