Virtual expression display method and device, storage medium and electronic equipment

Abstract

The invention discloses a virtual expression display method and device, a storage medium and electronic equipment. The method comprises the steps that a to-be-processed expression data set is obtained, and the expression data set is obtained according to facial expressions captured in real time in a real scene; based on the current data processing frequency indicated by the evaluation result of a comprehensive evaluation function, determining a first group of expression data of current batch processing from the expression data set, the comprehensive evaluation function being used for dynamically adjusting the data processing frequency and spatial processing parameters of batch processing; based on the current spatial screening parameters, screening the first group of expression data to obtain a second group of expression data to be displayed; and displaying a target virtual expression corresponding to the second group of expression data according to the current space display parameter. According to the method and the device, the technical problem of relatively low efficiency in a virtual expression display process is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the live broadcast technical field, in particular, a virtual expression display method and device, storage medium and electronic equipment are provided. BACKGROUND

[0002] With the development of Internet technology, real-time expression data display has become an important function cited by social media, live broadcast platforms and online meetings. In high concurrency scenarios, a large amount of expression data needs to be processed and displayed at the same time, which puts high requirements on the performance of mobile devices and network bandwidth.

[0003] In order to solve the above problems, in the related art, a timer mechanism is usually used to process the expression data received within a period of time in batches to reduce the performance overhead caused by frequent processing. However, this batch processing method according to a fixed time interval cannot dynamically adjust the processing frequency according to system load, network conditions and device performance, etc., so that processing delay may occur in high load conditions, thereby causing the technical problem of low efficiency of virtual expression display.

[0004] For the above problems, no effective solution has been proposed so far. SUMMARY

[0005] The embodiments of the present application provide a virtual expression display method and device, storage medium and electronic equipment to at least solve the technical problem of low efficiency in the process of displaying virtual expressions.

[0006] According to an aspect of an embodiment of the present application, a virtual expression display method is provided, comprising: obtaining a set of expression data to be processed, wherein the set of expression data is obtained according to real-time captured facial expressions in a real scene; determining a first group of expression data for current batch processing from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of batch processing; filtering the first group of expression data based on a current spatial filtering parameter to obtain a second group of expression data to be displayed, wherein the spatial processing parameters include the current spatial filtering parameter and the current spatial display parameter; and displaying a target virtual expression corresponding to the second group of expression data according to the current spatial display parameter.

[0007] Optionally, the current data processing frequency indicated by the evaluation result of the comprehensive evaluation function is used to determine the first group of expression data for the current batch processing from the expression data set, including: creating a comprehensive evaluation function based on a first evaluation function and a second evaluation function, wherein the first evaluation function is used to represent the processing efficiency and real-time performance of batch processing in the time dimension, and the second evaluation function is used to represent the visual effect coverage and spatial overlap rate of displaying virtual expressions in the space dimension; obtaining the historical evaluation result of the comprehensive evaluation function after the last batch processing; updating the parameter configuration information in the comprehensive evaluation function based on the historical evaluation result to obtain updated parameter configuration information, wherein the parameter configuration information includes data processing frequency, spatial processing parameter and preset weight coefficient; and obtaining the first group of expression data from the expression data set according to the time interval indicated by the current data processing frequency, wherein the updated parameter configuration information includes the current data processing frequency.

[0008] Optionally, the comprehensive evaluation function is created based on the first evaluation function and the second evaluation function, including: weighted sum of the first evaluation function, the second evaluation function and the space-time interaction term to obtain the comprehensive evaluation function, wherein the space-time interaction term is used to capture the nonlinear interaction relationship of the expression data in the expression data set in the time dimension and the space dimension.

[0009] Optionally, the parameter configuration information in the comprehensive evaluation function is updated based on the historical evaluation result to obtain the updated parameter configuration information, including: in the case that the historical evaluation result indicates that the batch processing queue has data backlog, increasing the data processing frequency and adjusting the first group of weight coefficients in the first evaluation function, wherein the batch processing queue includes the expression data set stored in order according to the priority of the expression data; and in the case that the historical evaluation result indicates that the spatial overlap rate of the virtual expression reaches a preset threshold, increasing the screening radius and adjusting the second group of weight coefficients in the second evaluation function.

[0010] Optionally, the first group of expression data is obtained from the expression data set according to the time interval indicated by the current data processing frequency, including: obtaining multi-modal load description information, wherein the multi-modal load description information includes network state data, device state data and online interaction data; determining the current data volume of the current batch processing based on the network state data, the device state data and the online interaction data; and obtaining the first group of expression data from the expression data set according to the time interval and the current data volume.

[0011] Optionally, the filtering, based on the current spatial filtering parameter, of the first set of expression data to obtain a second set of expression data to be displayed comprises: determining an initial weight and an initial priority of each expression data in the first set of expression data; saving the first set of expression data to a plurality of priority queues created in advance according to the initial priority; extracting expression data to be rendered from the plurality of priority queues based on the initial weight, wherein the virtual expression corresponding to the expression data is allowed to be displayed in a case where the initial weight is greater than a first threshold; locating potential conflict expressions having a spatial overlap rate greater than or equal to a second threshold with the virtual expression to be displayed based on a spatial index relationship, wherein the virtual expression to be displayed is obtained based on the expression data to be rendered; and filtering expression data corresponding to the potential conflict expressions according to a current filtering radius to obtain the second set of expression data, wherein the current spatial filtering parameter comprises the initial weight and the current filtering radius.

[0012] Optionally, the locating, based on the spatial index relationship, of potential conflict expressions having a spatial overlap rate greater than or equal to a second threshold with the virtual expression to be displayed comprises: obtaining a spatial index relationship between a spatial position and an interface display area, wherein the spatial position is position information of a virtual agent obtained according to a scene layout in an actual scene; obtaining an expression display density in the interface display area; adjusting an initial grid size based on the expression display density to obtain an adjusted grid size, wherein the initial grid size is preset, and the interface display area is divided into a group of grids according to the initial grid size; determining a current display area of the virtual expression to be displayed according to the spatial index relationship and the adjusted grid size; and determining adjacent grids connected with a grid occupied by the virtual expression to be displayed based on the current display area, and determining virtual expressions in the adjacent grids as the potential conflict expressions.

[0013] Optionally, the filtering, according to the current filtering radius, of expression data corresponding to the potential conflict expressions to obtain the second set of expression data comprises: obtaining an expression display density in the interface display area; determining the current filtering radius based on the expression display density and an initial filtering radius; filtering, according to the current filtering radius, expression data corresponding to the potential conflict expressions to obtain the second set of expression data; and the greater the current filtering radius, the greater the number of virtual expressions in the potential conflict expressions that are filtered out.

[0014] Optionally, the displaying the target virtual expression corresponding to the second set of expression data according to the current spatial display parameter comprises: determining a comprehensive weight of a set of virtual expressions corresponding to the second set of expression data based on a weight decay model, wherein the weight decay model is obtained based on a distance weight function between the set of virtual expressions, a time weight function based on a time of existence of the expression data, and an initial priority of the expression data; in a case where the comprehensive weight of each virtual expression in the set of virtual expressions satisfies a preset display condition, displaying the target virtual expression according to a display position and a transparency indicated by the current spatial display parameter; in a case where a first virtual expression and a second virtual expression in the set of virtual expressions have spatial overlap, and a difference between a first comprehensive weight of the first virtual expression and a second comprehensive weight of the second virtual expression is less than a third threshold value, adjusting a display position of the first virtual expression or interleaving the first virtual expression and the second virtual expression.

[0015] According to still another aspect of the embodiments of the present application, a virtual expression display device is further provided, comprising: a first acquisition unit configured to acquire a set of expression data to be processed, wherein the set of expression data is obtained according to a real-time captured facial expression in a real scene; a first processing unit configured to determine a first set of expression data for a current batch processing from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust a data processing frequency and a spatial processing parameter of the batch processing; a screening unit configured to screen the first set of expression data based on a current spatial screening parameter to obtain a second set of expression data to be displayed, wherein the spatial processing parameter comprises the current spatial screening parameter and a current spatial display parameter;

[0016] a first display unit configured to display a target virtual expression corresponding to the second set of expression data according to the current spatial display parameter.

[0017] According to still another aspect of the embodiments of the present application, a computer readable storage medium is further provided, and the computer readable storage medium stores a computer program, wherein the computer program is used to execute the virtual expression display method when the computer program is run by an electronic device.

[0018] According to still another aspect of the embodiments of the present application, a computer program product is further provided, and the computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the method.

[0019] According to still another aspect of the embodiments of the present application, an electronic device is further provided, and the electronic device comprises a memory and a processor, the memory stores a computer program, and the processor is configured to execute the virtual expression display method by the computer program.

[0020] By using the pre-created comprehensive evaluation function and the state data in the last batch processing process, the current data processing frequency and the current spatial screening parameter of the current batch processing are determined by using the above-mentioned embodiments provided in the application. According to the current data processing frequency, the first group of expression data is obtained from the expression data set to be processed, and the first group of expression data is screened according to the current spatial screening parameter to obtain the second group of expression data. Finally, the target virtual expression is displayed according to the current spatial display parameter. In other words, by optimizing the expression data processing performance in the time dimension and the space dimension, the optimal allocation of global resources is realized, so that the frequency of batch processing and the spatial screening result are dynamically adjusted according to the real-time situation, the smoothness and real-time performance of expression data display are ensured, and the technical effect of improving the display efficiency of virtual expression is realized. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are included to provide a further understanding of the application, form a part of the application and illustrate the illustrative embodiments of the application and together with the description serve to explain the application. The accompanying drawings are included as a part of the detailed description, to provide a further understanding of the application. The illustrative embodiments of the application and its description serve to explain the application. It does not constitute an inappropriate limitation on the application.

[0022] Figure 1 is a schematic diagram of an application scenario of an optional virtual expression display method according to an embodiment of the application;

[0023] Figure 2 is a flowchart of an optional virtual expression display method according to an embodiment of the application;

[0024] Figure 3 is a whole architecture diagram of an optional virtual expression display method according to an embodiment of the application;

[0025] Figure 4 is an implementation manner of an optional space-time joint optimization mechanism according to an embodiment of the application;

[0026] Figure 5 is a schematic diagram of an optional dynamic batch processing algorithm according to an embodiment of the application;

[0027] Figure 6 is a schematic diagram of an optional enhanced spatial screening algorithm according to an embodiment of the application;

[0028] Figure 7 is a display result of a virtual expression before improvement according to the technical solution of the application;

[0029] Figure 8 is a display result of a virtual expression after improvement according to the technical solution of the application;

[0030] Figure 9 is a structural schematic diagram of an optional virtual expression display device according to an embodiment of the application;

[0031] Figure 10 is a structural schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0034] The technical solutions in the embodiments of the present application will follow the laws during implementation, and the data used when performing operations according to the technical solutions in the embodiments will not involve user privacy. In addition to ensuring that the operation process is compliant and legal, the safety of the data is also ensured. In addition, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of relevant data need to comply with relevant regulations and standards in the relevant country or region.

[0035] In one aspect of the embodiments of the present application, a virtual expression display method is provided. As an optional implementation, the virtual expression display method can be applied to, but is not limited to, the application scenario as shown in Figure 1 Figure 1 ​In the application scenario shown, the target terminal 102 can communicate with the server 106 through the network 104, and the server 106 can perform operations on the database 108, such as data writing or reading. The target terminal 102 can include a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen can be used to display live pictures played by different video platforms on the target terminal 102. The processor can respond to the human-computer interaction operation, perform corresponding operations, or generate corresponding instructions, and send the generated instructions to the server 106. The memory stores processing data, such as an expression data set, a current data processing frequency, and a current spatial filtering parameter.

[0036] Optionally, in this embodiment, the target terminal can be a terminal configured with a target client, and can include at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a notebook computer, a tablet computer, a palm computer, a MID (Mobile Internet Device), a PAD, a desktop computer, a smart television, etc. The target client can be a video client, an instant messaging client, a browser client, an education client, etc. The network can include a wired network and a wireless network, wherein the wired network includes a local area network, a metropolitan area network, and a wide area network, and the wireless network includes Bluetooth, WIFI, and other wireless communication networks. The server can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0037] The technical solution of the present application can be widely applied to live streaming scenarios with high concurrency, limited resources, or poor network conditions. The following are specific examples of displaying virtual expressions in several application scenarios:

[0038] (1) By capturing the expression data of the audience in the actual viewing scene, according to the method in the present application, after receiving the expression data stream, the data processing frequency and the spatial processing parameter of the last batch processing are adjusted according to the comprehensive evaluation function to obtain the current data processing frequency and the current spatial processing parameter. According to the current data processing frequency of the current batch processing, a specific number of expression data are obtained from the data stream, and then the specific number of expression data are filtered according to the current spatial filtering parameter in the current spatial processing parameter. Finally, the filtered expression data are displayed according to the current spatial display parameter;

[0039] (2) Real-time comment section of social media platforms: During popular topics, public figures' updates, or major events, the comment section of social media platforms sees a surge in the sending of emojis, text, and images. The technical solution of this application can dynamically adjust the batch processing frequency and spatial filtering parameters, ensuring real-time display of emojis even under extremely high concurrency, avoiding interface lag and data loss, and improving user interaction experience.

[0040] (3) Online live streaming interaction: During live streaming, the frequency and quantity of user-sent emojis are usually unpredictable, especially in multi-host or large-scale event live streaming scenarios. The technical solution of this application can adjust the processing and display strategy of emojis according to real-time network conditions and device performance, effectively preventing emoji delays in weak network environments and interface lag when device performance is limited, ensuring smoothness of live streaming interaction.

[0041] (4) Online conference system: With the increasing demand for remote collaboration, emojis have been introduced into online conferences to enhance emotional expression in communication. In large-scale conferences with a large number of participants, the amount of emoji data is large. The technical solution of this application can dynamically adjust the emoji processing strategy according to the number of participants and network conditions through the spatio-temporal optimization mechanism, ensuring that even in complex network environments, emoji data can be efficiently processed and clearly displayed, improving conference participation and interaction experience.

[0042] In summary, the technical solution of this application can effectively filter and display real-time emoji data of live audience in high-concurrency and complex viewing scenarios, establishing an instant connection between audience and performers, greatly enriching the interactivity and immersion of live streaming, and testing the stability and performance optimization capabilities of the system in processing a large amount of real-time video input. Not only can it improve audience participation and satisfaction, but also can add unique viewing value to sports broadcasting or live performances, promoting the dissemination and social sharing of live streaming content.

[0043] As described in the above embodiments, the traditional virtual emoji display method is prone to low efficiency. To solve this problem, the virtual emoji display method in the present application is proposed, Figure 2 is a flowchart of the virtual emoji display method according to the present application, which includes the following steps S202-S208.

[0044] It should be noted that the virtual emoji display method shown in steps S202-S208 can be executed by an electronic device, but is not limited to, which can be, but is not limited to, a target terminal or a server as shown in Figure 1 .

[0045] In step S202, a set of expression data to be processed is obtained, wherein the set of expression data is obtained according to real-time capturing of facial expressions in a real scene;

[0046] In step S204, a first group of expression data for current batch processing is determined from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of batch processing;

[0047] In step S206, the first group of expression data is filtered based on a current spatial filtering parameter to obtain a second group of expression data to be displayed, wherein the spatial processing parameters include the current spatial filtering parameter and a current spatial display parameter.

[0048] In step S208, a target virtual expression corresponding to the second group of expression data is displayed according to the current spatial display parameter.

[0049] Before explaining the technical solutions in the embodiment, two implementation methods of real-time expression data display in the related art are briefly introduced.

[0050] The first method is batch processing technology, which processes expression data received in a period of time to reduce performance overhead caused by frequent processing. A common implementation method is to use a double timer architecture, one timer is responsible for collecting data, and the other timer is responsible for processing data. However, this fixed time interval batch processing method cannot dynamically adjust according to system load, network conditions and device performance, and in high load conditions, it is easy to cause processing delay, and in low load conditions, it will cause resource waste.

[0051] The second method is spatial filtering technology: by judging the spatial position of expression data, the data to be displayed is filtered out to avoid performance pressure caused by full processing. Traditional spatial filtering usually uses an N x N conflict detection algorithm to make a binary judgment (in / out of the filtering range) for each expression data, which has high computational complexity and cannot distinguish the importance of data, which may cause important expression data to be filtered out.

[0052] That is, in the related art, the display of expression data in a complex scene is usually processed by using a time dimension or a spatial dimension processing strategy, and there is a lack of collaborative optimization mechanism between the two, which cannot achieve optimal allocation of global resources. Moreover, the batch processing and spatial filtering parameters are usually statically preset and cannot be dynamically adjusted according to real-time conditions, which is difficult to adapt to complex and variable application scenarios; in a weak network environment or a device performance limited condition, the above method of processing expression data in one dimension cannot guarantee the smoothness and real-time performance of expression data display, which may cause problems such as lag, delay or loss

[0053] To solve the above problems, the embodiment proposes a real-time expression data display method based on spatio-temporal joint optimization, to solve the problems of independent processing of time dimension and space dimension, static preset of parameter configuration, and improve the fluency, real-time performance and resource utilization of expression data display.

[0054] Specifically, a three-level processing architecture as shown in Figure 3 may be deployed on the client side, wherein the data receiving module is responsible for the acquisition and preprocessing of the original expression data, the spatio-temporal joint optimization engine integrates the improved dynamic batch processing and enhanced spatial filtering algorithm, and the emotion mapping rendering module renders and displays the filtered data as expression bubbles.

[0055] The expression data set includes but is not limited to the expression data (such as expression ID, timestamp, spatial position, etc.) corresponding to the real-time capture of the audience's facial expressions in the real scene by the camera, and is converted into an expression data stream for transmission to the expression data processing system, such as the system architecture as shown in Figure 3 .

[0056] The above comprehensive evaluation function can be but not limited to the first evaluation function corresponding to the improved batch processing mechanism, the second evaluation function corresponding to the enhanced spatial filtering algorithm, and the spatio-temporal interaction term. The comprehensive evaluation function is used to measure the cooperative optimization effect of the time dimension and the space dimension in real time. For example, the time dimension mainly includes data processing frequency, batch processing data size (which can also be understood as data size), etc., and the space dimension mainly includes spatial filtering radius, weight coefficient, and spatial display parameters, etc.

[0057] The data processing frequency can be but not limited to the time interval of obtaining the expression data of each batch processing from the batch processing queue. For example, 500 expression data are selected from the queue for the first batch processing every 1s, and then 550 expression data are selected from the queue for the second batch processing after an interval of 3s, etc.

[0058] The spatio-temporal interaction term is used to capture the nonlinear interaction relationship between the time dimension and the space dimension, which will be explained and described in combination with specific embodiments.

[0059] That is, the comprehensive evaluation function (which can also be understood as a comprehensive scoring function) is determined according to the improved dynamic batch processing mechanism, the enhanced spatial filtering algorithm, and the spatio-temporal joint optimization mechanism.

[0060] Among them, the improved dynamic batch processing mechanism is an improved dynamic batch processing mechanism with load awareness. The mechanism dynamically adjusts the batch size and frequency by monitoring the system load, network conditions and device performance in real time. When the system load is low, increase the batch size to improve throughput; when the system load is high, reduce the batch size to reduce latency. At the same time, the priority mechanism of expression data is introduced to ensure that important expression data (such as anchors, senior users, etc.) are processed first, improving user experience.

[0061] The enhanced spatial screening algorithm associates the display priority of expression data with its distance from the center point, the newness of the generation time, user attention and interaction frequency, etc. The final display result is determined by weight calculation. At the same time, the system dynamically adjusts the screening radius according to the device performance and the current expression density, optimizing the performance while ensuring the visual effect.

[0062] The spatio-temporal joint optimization mechanism mainly realizes the bidirectional feedback mechanism of batch processing and spatial screening through the establishment of a unified scoring function (which can also be understood as a comprehensive evaluation function), and its purpose is to realize the optimization of global resource allocation.

[0063] Specifically, spatio-temporal joint optimization mainly links the time dimension (batch processing) and the space dimension (screening) through the comprehensive evaluation function. The form of the comprehensive evaluation function is as follows formula (1):

[0064] Score(t, s) = ω1·f(t) + ω2·g(s) + ω3·h(t, s) (1)

[0065] Where t and s represent the parameters of the time dimension and the space dimension respectively, f(t) is the first evaluation function (time dimension score), which is used to measure the efficiency and real-time performance of batch processing; g(s) is the second evaluation function (space dimension score), which is used to measure the effect and display quality of spatial screening; h(t,s) is the space-time interaction term, which is used to measure the synergistic effect of the time dimension and the space dimension. ω1, ω2, ω3 are weight coefficients, and ω1+ω2+ω3=1. The system adjusts these parameters continuously to achieve global optimal allocation of resources, solving the local optimal problem caused by independent optimization of each module in traditional methods.

[0066] As shown in Figure 4 The specific process of dynamically adjusting the data processing frequency of batch processing and the space processing parameters in the spatial screening algorithm using the comprehensive evaluation function is as follows:

[0067] S11, data initialization;

[0068] Mainly including the creation of comprehensive evaluation function, and set the initial weight coefficient, create global resource pool and resource allocation strategy, the initialization performance index collection system and the establishment of time dimension and space dimension communication channel, etc.

[0069] S12, the performance index of time dimension and space dimension is periodically evaluated;

[0070] For example, by collecting the performance index of two dimensions, the function value of the current comprehensive evaluation function (which can also be understood as the same scoring function) is calculated, and the gradient direction of the scoring function is analyzed.

[0071] S13, according to the gradient direction of the comprehensive scoring function, adjust the resource allocation proportion, and update the parameter configuration of batch processing and space screening algorithm;

[0072] That is, according to the evaluation result of the comprehensive scoring function, the weight coefficient in the above formula (1) is adjusted, and the configuration parameters in the time dimension and the space processing parameters in the space screening algorithm in the last batch processing are updated to obtain the current processing parameters (which can also be understood as the evaluation result) for the current batch processing.

[0073] Among them, in the process of global resource allocation, according to the gradient direction of the comprehensive evaluation function, the resource proportion allocated to the time dimension and the space dimension is dynamically adjusted. At the same time, the resource usage of each dimension is monitored in real time to avoid the occurrence of resource shortage and excessive occupation.

[0074] S14, according to the evaluation result, the parameters in the space dimension and the time dimension are optimized;

[0075] S14-1, performance index collection;

[0076] Mainly including the queue length, processing delay and batch processing efficiency in time dimension; Conflict rate, display quality, user interaction response in space dimension; System state indicators include CPU usage, memory occupancy, frame rate and network status, etc.

[0077] S14-2, adaptive adjustment of parameters;

[0078] Mainly including batch processing parameters: batch size, processing interval (data processing frequency) and queue priority; Space screening parameters: screening radius, weight coefficient and grid size; Evaluation function parameters: based on historical data and current trend, predict system load change, adjust parameters in advance.

[0079] For example, when the evaluation result indicates that the space conflict rate is too high, the parameters in the space dimension are adjusted; when the evaluation result indicates that the batch processing queue is accumulated, the parameters in the time dimension are adjusted.

[0080] S15, record the parameter adjustment result, and optimize the weight coefficient.

[0081] After determining the evaluation result of the comprehensive evaluation function according to the above method, and performing the improved batch processing mechanism and the spatial screening algorithm according to the evaluation result, the batch processing and spatial screening of the expression data are completed, and the target virtual expression corresponding to the screened expression data (i.e., the second group of expression data) is displayed.

[0082] After completing the batch processing and spatial screening, the system will render and display the second group of expression data in the live or social interface based on the current spatial display parameters (such as the minimum display interval of the expression, transparency control, etc.), to achieve a clear and non-overlapping visual effect. For example, the spatial display parameters may specify that a maximum of 1000 expressions are allowed to be displayed on the screen at the same time, and the minimum interval between expressions is a preset value. Then, the system will perform the final layout and rendering of the second group of expression data according to these parameters, to ensure that each expression is properly displayed on the UI interface.

[0083] In addition, during the processing of the spatio-temporal joint optimization mechanism, the following abnormal situations may occur:

[0084] (1) Resource competition processing: when two dimensions conflict in demand for the same resource, the priority is determined by the scoring function; (2) Performance bottleneck handling: identify system bottlenecks and dynamically adjust the allocation strategy of limited resources; (3) Graceful degradation mechanism: in extreme cases, degrade the function according to the preset strategy to ensure the core experience.

[0085] In the above manner, the current data processing frequency and the current spatial screening parameter of the current batch processing are determined by using the pre-created comprehensive evaluation function and the state data in the last batch processing process. The first group of expression data is obtained from the set of expression data to be processed according to the current data processing frequency, and the second group of expression data is obtained by screening the first group of expression data according to the current spatial screening parameter. Finally, the target virtual expression is displayed according to the current spatial display parameter. In other words, by optimizing the expression data processing performance in the time dimension and the space dimension, the optimal allocation of global resources is achieved, so that the frequency of batch processing and the spatial screening result are dynamically adjusted according to the real-time situation, ensuring the smoothness and real-time performance of expression data display, and achieving the technical effect of improving the display efficiency of virtual expressions.

[0086] As an optional example, the current data processing frequency indicated by the evaluation result of the comprehensive evaluation function is determined from the set of expression data, including:

[0087] create a comprehensive evaluation function based on the first evaluation function and the second evaluation function, wherein the first evaluation function is used to represent the processing efficiency and real-time performance of performing batch processing in the time dimension, and the second evaluation function is used to represent the visual effect coverage and spatial overlap rate of displaying virtual expressions in the space dimension;

[0088] obtain a historical evaluation result of the comprehensive evaluation function after the last batch processing;

[0089] update the parameter configuration information in the comprehensive evaluation function based on the historical evaluation result to obtain updated parameter configuration information, wherein the parameter configuration information includes data processing frequency, spatial processing parameters, and a preset weight coefficient;

[0090] obtain a first set of expression data from the expression data set according to a time interval indicated by the current data processing frequency, wherein the updated parameter configuration information includes the current data processing frequency.

[0091] As can be known from the description in the above embodiments, the comprehensive evaluation function is determined based on the first evaluation function and the second evaluation function and the space-time interaction term, wherein the first evaluation function can be but is not limited to the following formula (2):

[0092] f(t)=α1·(1 / L)+α2·(1 / D)+α3·(1 / M) (2)

[0093] wherein L is the queue length of the current batch processing queue, D is the processing delay, M is the memory occupancy rate, and α1, α2, and α3 are the internal weights in the time dimension.

[0094] The second evaluation function can be but is not limited to the following formula (3):

[0095] g(s)=β1·V+β2·(1-C)+β3·Q (3)

[0096] wherein V is the visualization coverage (which can also be understood as the ratio between the number of expressions allowed to be displayed and the total number of expressions in the current batch processing), C is the conflict rate (which can also be understood as the spatial overlap rate between expressions to be displayed), C is the display quality score, and β1, β2, and β3 are the internal weights in the space dimension.

[0097] After determining the first evaluation function and the second evaluation function, the comprehensive evaluation function can be but is not limited to being determined in the following manner:

[0098] perform weighted summation on the first evaluation function, the second evaluation function, and the space-time interaction term to obtain the comprehensive evaluation function, wherein the space-time interaction term is used to capture the nonlinear interaction relationship of the expression data in the expression data set in the time dimension and the space dimension.

[0099] The spatiotemporal interaction term can be, but is not limited to, as shown in the following formula (4):

[0100] h(t, s) = γ1·(t·s) + γ2·min(t, s) + γ3·max(t, s) (4)

[0101] Where γ1, γ2, γ3 are the internal weights of the interaction term.

[0102] As shown in Figure 4 The current data processing frequency in the current batch processing process is obtained by updating the historical data processing frequency after the last batch processing according to the gradient direction of the comprehensive evaluation function (historical evaluation result).

[0103] For example, when the current batch queue is congested, the batch processing frequency is increased, for example, the time interval for obtaining expression data in the last batch processing process is 3s, then after 1s, the expression data for the current batch processing is obtained from the queue to perform batch processing and spatial screening.

[0104] That is, according to the comprehensive evaluation function and the historical evaluation result, the data processing frequency of the current batch processing is dynamically adjusted. At the same time, when the system load is low, the batch size is increased to improve the throughput; when the system load is high, the batch size is reduced to reduce the delay.

[0105] By fusing the first evaluation function and the second evaluation function to construct a comprehensive evaluation function, the collaborative optimization of time dimension and space dimension is realized, and the local optimal trap caused by traditional single dimension optimization is broken. The acquisition and analysis of the historical evaluation result further enhances the adaptability and intelligent optimization capability of the system, so that the parameter configuration can be dynamically adjusted according to the real-time environmental changes, and the system can play the best performance in different scenarios.

[0106] The dynamic parameter adjustment mechanism is the key to improve the efficiency of expression data processing and display. By analyzing the historical evaluation result and updating the parameter configuration information accordingly, the system can flexibly cope with various environmental challenges, including high-concurrency data flow, weak network environment and performance-limited devices. This mechanism ensures that the batch processing and spatial screening parameters can match the system load in real time, which not only significantly reduces the data processing delay, but also reduces the memory occupation and GPU peak load.

[0107] The update strategy of the parameter configuration information adopts a method combining a feedback loop and predictive adjustment. The feedback mechanism automatically adjusts the batch processing frequency, spatial processing parameters, and weight coefficients based on historical evaluation results by analyzing the fluctuation trend of performance indicators to optimize the overall performance of the system. The predictive adjustment predicts future performance requirements based on historical data and the current system state and adjusts the parameter configuration in advance, such as actively reducing the batch processing frequency and screening radius before the network quality decreases to ensure continuous display of expression data and reduce stuttering and data loss caused by network fluctuations.

[0108] The intelligent adjustment of the above data processing frequency is a key strategy of the present application to improve the system response capability and user experience smoothness. By monitoring the system state and combining historical evaluation results, the system can automatically adjust the data processing frequency to ensure that batch processing can meet the data processing needs in high-concurrency scenarios and reasonably allocate computing resources based on the current device performance and network conditions to avoid resource waste and performance bottlenecks, achieving the best balance between resource utilization and user experience.

[0109] By constructing a comprehensive evaluation function and dynamically adjusting the parameter configuration information based on historical evaluation results, the expression data is optimized in the time and space dimensions. Not only does this significantly improve the efficiency of expression data processing and the visual effect of expression data display, but it also enhances the adaptability and intelligence level of the system, providing users with a smooth and high-quality real-time expression data display experience in various complex environments and application scenarios.

[0110] As an optional example, the above updating the parameter configuration information in the comprehensive evaluation function based on the historical evaluation results to obtain updated parameter configuration information includes:

[0111] In the case where the historical evaluation results indicate that the batch processing queue has data backlog, the data processing frequency is increased, and the first set of weight coefficients in the first evaluation function is adjusted, wherein the batch processing queue includes a set of expression data stored in order of priority.

[0112] In the case where the historical evaluation results indicate that the spatial overlap rate of virtual expressions reaches a preset threshold, the screening radius is increased, and the second set of weight coefficients in the second evaluation function is adjusted.

[0113] In the implementation process of the technical solution of the present application, the system continuously monitors the state of the batch processing queue to ensure that data can be processed in a timely and effective manner. If the batch processing queue has a data backlog, meaning that the expression data waiting to be processed in the queue exceeds the normal level, this may be due to high traffic data input, device performance limitations, or poor network conditions. In this case, the system will take appropriate measures to alleviate the backlog problem. For example, increase the data processing frequency and adjust the weight coefficient in the first evaluation function.

[0114] For example, assume that during a live event in a peak period, the data backlog of the batch processing queue reaches a historical high, with the queue length reaching 5000 expression data, far exceeding the threshold of 1000. At this time, the data processing frequency is modified from the original 1 batch processing every 3 seconds to 1 batch processing every 1 second to process the data in the queue more quickly. At the same time, the system also adjusts the weight coefficient in the first evaluation function, giving more attention to batch processing efficiency to ensure that real-time performance is not affected.

[0115] Conversely, if the historical evaluation result indicates that the spatial overlap rate of virtual expressions has reached or exceeded the preset threshold, it means that the current parameter configuration of the spatial screening algorithm is no longer suitable for the current scenario, resulting in a decline in expression display effect. At this time, the system will increase the screening radius and adjust the weight coefficient in the second evaluation function accordingly to reduce spatial overlap and improve the coverage rate of visual effects.

[0116] For example, assume that in an online meeting with high interaction frequency, the number of virtual expressions sent by participants has increased dramatically, resulting in excessive density of expressions on the screen, with a spatial overlap rate of 80%, far exceeding the preset threshold of 30%. The system immediately increases the screening radius to allow fewer expressions to be displayed in the same area, while adjusting the weight coefficients in the second evaluation function related to spatial coverage and conflict rate to give them higher weights, ensuring the clarity and visual effect of expression display.

[0117] As can be seen, when dealing with data backlog and spatial overlap problems using the technical solution in the present embodiment, the system can intelligently identify and respond to possible data backlog conditions in the batch processing queue by analyzing historical evaluation results, increasing data processing frequency, and adjusting the weight coefficient in the first evaluation function, ensuring that expression data can be quickly processed in high-concurrency scenarios, reducing user waiting time, and improving the real-time performance and smoothness of user experience.

[0118] Meanwhile, attention is also paid to the expression display effect in the spatial dimension, especially in the case of a too high virtual expression space overlap rate. By increasing the filtering radius and adjusting the weight coefficient in the second evaluation function, the system can effectively reduce the overlap between expressions, improve the visual effect of expression display and the aesthetic sense of user experience, and avoid user interface confusion and visual fatigue caused by unreasonable spatial layout.

[0119] The embodiments of the present application not only solve the performance bottleneck and visual layout problems existing in traditional expression data processing, but also greatly improve the adaptability and intelligent optimization capability of the system in the face of complex and variable application environment. It ensures that the processing and display of expression data can achieve the best balance state under load conditions, and can make timely and effective response to both high-concurrency data input and fine requirements for spatial layout.

[0120] As an optional example, the time interval according to the current data processing frequency indication is used to obtain the first group of expression data from the expression data set, which includes:

[0121] Obtain multi-modal load description information, wherein the multi-modal load description information includes network state data, device state data and online interaction data;

[0122] Determine the current data volume of the current batch processing based on the network state data, the device state data and the online interaction data;

[0123] Obtain the first group of expression data from the expression data set according to the time interval and the current data volume.

[0124] The specific implementation of the improved dynamic batch processing mechanism in the technical scheme of the present application can refer to Figure 5 As shown in the figure, it includes initialization phase and running phase two parts, wherein, the initialization phase mainly sets the basic batch processing parameters (such as batch size, initial processing frequency), initializes the monitoring system and establishes the data acquisition channel, creates the priority queue and batch processing buffer zone, etc.

[0125] In the running phase, the following processing process is mainly executed:

[0126] S21, multi-dimensional monitoring system;

[0127] The multi-dimensional monitoring system can but not limited to obtain multi-modal load description information, wherein the multi-modal load description information includes but not limited to the following aspects:

[0128] (1) Network status monitoring: real-time collection of network delay, bandwidth and packet loss rate, calculation of network quality index (NQI) through sliding window algorithm;

[0129] (2) Device performance monitoring: periodically collect CPU usage, memory occupation and GPU load to construct a device performance index (DPI);

[0130] (3) User interaction monitoring: record user operation frequency, operation type and operation interval to generate a user interaction activity index (UAI).

[0131] S22, according to the above multi-modal load description information, adaptively adjust the batch size (i.e. the amount of data in one batch, such as 500 expression data or 600 expression data), which is realized by the following formula (5):

[0132] BatchSize = BaseBatchSize x a x (NQI + DPI + UAI) (5)

[0133] Wherein, a is a weight coefficient.

[0134] The batch frequency can be dynamically adjusted by the following formula (6):

[0135] Interval = BaseInterval / (β * (NQI + DPI + UAI)) (6)

[0136] Wherein, β is a frequency adjustment coefficient.

[0137] In the improved batch processing process in the technical scheme of the present application, a threshold adaptive mechanism is also set, which mainly triggers emergency batch processing when the monitoring index exceeds the preset threshold (such as the number of data in the queue exceeds the threshold), to ensure the real-time performance of data.

[0138] S23, priority queue management;

[0139] (1) Data classification: classify expression data into high, medium and low levels according to importance, and assign weight coefficients respectively;

[0140] (2) Priority calculation: consider data timeliness, user attention and content relevance to calculate the priority score of each data;

[0141] (3) Queue scheduling: high priority data is given priority to enter the batch processing queue to ensure that important data is processed first.

[0142] S24, feedback adjustment mechanism.

[0143] (1) Performance feedback: monitor the system performance change after each batch processing, and adjust the next batch processing parameters;

[0144] For example, collect performance data of this batch to update NQI, DPI, and UAI indexes, and update data volume (batch size) BatchSize and processing frequency Interval for the next batch. In addition, the priority calculation weight will also be adjusted.

[0145] (2) User experience feedback: Collect UI rendering frame rate, interaction response time, etc. as the basis for adjusting batch parameters.

[0146] This embodiment mainly adjusts the operation parameters of batch processing, such as data volume and time interval, according to multi-modal load description information. By monitoring the network, device, and user online interaction in real time, the optimal data volume of the current batch can be intelligently determined, so as to avoid excessive resource consumption or performance bottlenecks while ensuring data processing efficiency. It ensures that the system can respond flexibly in various complex scenarios.

[0147] For example, in the case of large network environment fluctuations or limited device performance, the system will automatically reduce the batch data volume and extend the processing interval to avoid delays or lag caused by excessive data processing pressure. On the contrary, in the case of stable network, sufficient device performance, and frequent user interaction, the system will increase the batch data volume and shorten the processing interval to fully utilize resources and improve the throughput and real-time performance of data processing.

[0148] In addition, the following exception handling mechanism can be used to handle some abnormal situations during processing according to the improved dynamic batch processing algorithm in the embodiments of the present application:

[0149] (1) Network fluctuation response: when the network quality drops sharply, reduce the batch size and increase the processing frequency;

[0150] (2) Device overload protection: when the device performance index is below the safety threshold, suspend non-critical data processing to ensure core functions;

[0151] (3) Queue overflow control: when the queue length approaches the upper limit, start the emergency processing mode, which may discard low-priority data.

[0152] Through the above-mentioned manner, the batch processing operation can be intelligently adjusted according to the changes of the real-time environment, which not only improves the overall efficiency of expression data processing, but also optimizes the user experience. Especially in the scenario of high concurrency of expression data, it can ensure performance while providing users with a smoother and more immediate interactive experience, which is suitable for various application scenarios that require real-time expression display.

[0153] As an optional example, the above filtering of the first group of expression data based on the current space filtering parameters to obtain the second group of expression data to be displayed includes:

[0154] determine an initial weight and an initial priority of each expression data in the first group of expression data;

[0155] save the first group of expression data to a plurality of priority queues created in advance according to the initial priority;

[0156] extract expression data to be rendered from the plurality of priority queues based on the initial weight, wherein the virtual expression corresponding to the expression data is allowed to be displayed when the initial weight is greater than a first threshold value;

[0157] locate potential conflict expressions with a spatial overlap rate greater than or equal to a second threshold value with the virtual expression to be displayed based on the spatial index relationship, wherein the virtual expression to be displayed is obtained based on the expression data to be rendered;

[0158] screen the expression data corresponding to the potential conflict expressions according to a current screening radius to obtain a second group of expression data, wherein the current spatial screening parameter includes the initial weight and the current screening radius.

[0159] After determining the first group of expression data of the current batch according to the method in the above embodiment, the first group of expression data is screened by using an enhanced spatial screening algorithm, so as to determine the expression data that can be finally displayed and the display effect.

[0160] In this embodiment, by introducing a weight decay model and a priority queue, dynamic screening based on weight is realized, and the specific implementation manner can refer to the schematic diagram shown in Figure 6 The detailed steps are as follows:

[0161] S31, create a weight decay model;

[0162] S31-1, construct a distance weight function;

[0163] An improved Gaussian decay function as shown in the following formula (7) can be used, but is not limited to:

[0164] W(d)=e^(-(d 2 / 2σ 2 )) (7)

[0165] wherein d is the distance between two expressions, and σ is an adjustable decay coefficient, controlling the speed of the weight decay with the increase of distance. When the distance between two expressions is small, the value of W(d) is close to 1, indicating that the spatial influence between the two expressions is relatively large, and the two expressions can be displayed simultaneously and will not be filtered out due to distance. As the distance between the two expressions gradually increases, the value of W(d) gradually decreases, indicating that the spatial influence between the two expressions is weakened, which means that the possibility of being filtered out or position adjusted by the filter is increasing.

[0166] The introduction of the distance weight function helps to reasonably distribute expressions in space, avoid overlapping and visual confusion. At the same time, it can also ensure that the expressions in key positions (such as around the host) are displayed preferentially, thereby optimizing the visual effect of expression data display.

[0167] S31-2, constructing a time weight function;

[0168] Specifically, a time decay factor as shown in the following formula (8):

[0169] T(t) = e^(-λt) (8)

[0170] wherein t is the existence time of expression data, and λ is a time decay coefficient. The existence time of expression data can be, but is not limited to, the starting time from capturing the expression, or the duration from the starting time to the current time.

[0171] S31-3, constructing a weight decay model for comprehensive weight calculation.

[0172] Specifically, as shown in the following formula (9):

[0173] Weight = α·W(d) + β·T(t) + γ·P (9)

[0174] wherein α, β, γ are adjustable weight coefficients, and P is the inherent priority of the expression.

[0175] S32, implementation of priority queue;

[0176] S32-1, creating a multi-level priority queue;

[0177] For example, three levels of high, medium and low priority queues are established, respectively corresponding to expression data of different importance;

[0178] S32-2, dynamic adjustment of priority;

[0179] According to user interaction, content popularity and space-time factors, the priority of expression data is periodically recalculated.

[0180] S32-3, queue scheduling strategy;

[0181] The data is selected from the queues for processing in a weighted round robin manner.

[0182] S33, space index optimization;

[0183] S33-1, grid partition index;

[0184] The display area can be divided into m x n grids, and each expression is mapped to the corresponding grid according to the seat position in the real scene. That is, each real seat corresponds to a grid displayed on the interface, and m and n are positive integers greater than or equal to 2.

[0185] S33-2, adjacent grid detection;

[0186] The expression in the current network and adjacent grid can be detected, and the complexity can be reduced from O(n 2 ) to O(k·n), where k is a constant.

[0187] S33-3, dynamic grid adjustment.

[0188] The grid size is dynamically adjusted according to the expression density, for example, large grid is used in sparse area and small grid is used in dense area.

[0189] S34, setting of dynamic screening radius.

[0190] First, the initial screening radius (which can also be understood as the basic screening radius) is set, that is, the default minimum distance requirement R0 between expressions is set. The dynamic adjustment of the screening radius is realized according to the following formula (10):

[0191] R = R0* (1-δ·D) (10)

[0192] Where D is the expression density of the current area, and δ is the adjustment coefficient.

[0193] Based on the above formula (10), an adaptive threshold can be used to determine whether to allow the expression to be displayed. For example, when the weight value is greater than or equal to the pre-set threshold, the expression is allowed to be displayed; when the threshold is lower, the expression is screened out or the expression display position is adjusted.

[0194] In one specific example, the space screening is performed by the following steps:

[0195] S41, assuming that the initial weight and initial priority of each expression data are calculated according to the inherent attributes of the expression data (such as the type of expression, user level, generation time, etc.);

[0196] The initial weight reflects the importance of the expression data, and the initial priority further guides the processing and display order of the expression data.

[0197] For example, when the system receives a new set of expression data, such as "applause", "like", "surprise" and other expressions sent by the audience in the live broadcast, the system will assign an initial weight to the expression according to the type of the expression and the level of the sending user. For the "applause" expression, since its positive emotional value is high and the sending user is VIP, the system will give a higher initial weight; while the "surprise" expression, due to the low emotional intensity and user level, its initial weight is correspondingly low.

[0198] S42, according to the calculated initial priority, the system stores the expression data into a plurality of priority queues created in advance. These queues are sorted in descending order of priority, ensuring that high-priority expression data can be processed and displayed first;

[0199] Still taking the above example for explanation, the system stores the "applause" expression in the high-priority queue, and the "surprise" expression in the low-priority queue. This storage strategy ensures that important expression data can be processed in time, improving the quality of user experience.

[0200] S43, preliminary screening based on the initial weight of the expression;

[0201] Among them, only when the initial weight of the expression exceeds the first threshold value, the expression data is allowed to be further processed, and finally displayed as a virtual expression in the user interface.

[0202] Among them, the initial weight can be but not limited to determined in the current batch processing process, or obtained by updating the historical weight in the last batch processing and the last spatial screening algorithm.

[0203] For example, assuming that the system sets the first threshold value to 0.5, and the initial weight of the "applause" expression is 0.7, then the expression data will pass the preliminary screening and enter the next processing flow; if the initial weight of the "surprise" expression is 0.3, the system will not process the expression data temporarily, thereby effectively controlling the number of expression displays and avoiding excessive crowding of the UI interface.

[0204] S44, conflict detection and optimization based on spatial index relationship.

[0205] Potential conflicting expressions (i.e. expressions that may overlap) are located through spatial index relationships, i.e. the spatial overlap rate between the expression to be displayed and the displayed expression exceeds the second threshold value. For these potential conflicting expressions, the system further screens according to the current screening radius and the initial weight of the expression to determine the final set of expression data to be displayed.

[0206] For example, on the UI interface, multiple "like" expressions appear at the same time, which may cause spatial overlap display. Through spatial indexing, it is detected that the spatial overlap rate between these expressions exceeds the set second threshold (such as 0.6). For the overlapping expressions, the system will determine which expressions can continue to be displayed and which expressions need to adjust the position or reduce the transparency according to their initial weights and the currently set filtering radius, to achieve the best visual effect and performance balance.

[0207] In an optional embodiment, assuming that the virtual expressions displayed on the UI interface are not batch-processed and spatially filtered according to the technical solutions of the present application, as shown in FIG. 1A, a large area of accumulation and overlap appears. After batch-processing and spatial filtering according to the technical solutions of the present application, part of the expression data is adaptively selected from the expression data and displayed on the UI interface as shown in FIG. 1B. Figure 7 As shown in FIG. 1B, the virtual expressions are distributed in different areas on the interface, and neither overlap between expressions nor redundant display of the same or similar expressions occurs, improving the visual effect and user experience. Figure 8 Figure 8 As shown in FIG. 1B, the virtual expressions are distributed in different areas on the interface, and neither overlap between expressions nor redundant display of the same or similar expressions occurs, improving the visual effect and user experience.

[0208] In addition, during the processing according to the enhanced spatial filtering algorithm in the embodiments of the present application, the following conflict resolution strategies may occur:

[0209] (1) Position fine-tuning: when a spatial conflict is detected but the weight is close, fine-tuning the position is used to avoid complete filtering out;

[0210] (2) Transparency adjustment: for expressions with low weight but not completely filtered out, the transparency is reduced to reduce visual conflict;

[0211] (3) Time-sharing display: for conflicting expressions with similar weights, time-interleaved display is used to avoid simultaneous display.

[0212] In the above manner, by calculating the initial weight and priority of the expression data, the processing order and display or not of the expression data can be intelligently determined. The important expression data is ensured to be processed in priority, the invalid occupation of system resources is reduced, and the processing efficiency is improved. Through the use of the priority queue, the system can quickly extract the data according to the importance of the expression data, avoid processing delay, and improve the response speed of the user interface. Based on the conflict detection and optimization strategy of the spatial indexing relationship, the system can finely control the display of the expressions, avoid the overlap of the UI interface, and provide a better visual experience.

[0213] As an optional implementation manner, the above potential conflicting expressions positioned and to be displayed in the virtual expression based on the spatial indexing relationship include:​

[0214] obtaining a spatial index relationship between a pre-created spatial position and an interface display area, wherein the spatial position is position information of a virtual agent obtained according to a scene layout in an actual scene;

[0215] obtaining an expression display density in the interface display area;

[0216] adjusting an initial grid size based on the expression display density to obtain an adjusted grid size, wherein the initial grid size is pre-set, and the interface display area is divided into a group of grids according to the initial grid size;

[0217] determining a current display area of a virtual expression to be displayed according to the spatial index relationship and the adjusted grid size;

[0218] determining a neighboring grid connected to a grid occupied by the virtual expression to be displayed based on the current display area, and determining a virtual expression in the neighboring grid as a potential conflict expression.

[0219] Through the pre-constructed spatial index relationship, the virtual position of the expression data in the actual scene can be quickly located. This position information is derived from scene layout analysis. For example, in a live performance, the positions (or seats) of each viewer and each performer are pre-arranged. Through the spatial index relationship between each position and the display interface, the screened expression data is mapped to the grid where the UI interface is located, and the expression data is displayed in the area where the grid is located.

[0220] After determining the display position of the expression data, the expression display density in the interface display area is calculated. This index reflects the number of expressions in a unit area, which is used for subsequent spatial optimization decisions.

[0221] Based on the current expression display density, the system will dynamically adjust the initial grid size to optimize the spatial distribution of expression data. When the expression density is high, the system will reduce the grid size, making the grid more detailed, which facilitates more accurate control of expression position and avoids overlapping. Conversely, when the expression density is low, the grid size is increased to reduce the number of grids, thereby reducing the computational complexity of spatial screening.

[0222] In the process of displaying expression data, the current display area of the virtual expression to be displayed is determined according to the adjusted grid size and the spatial index relationship, and the expressions in the neighboring grid that may have a spatial conflict are identified. This mechanism helps to reduce the occlusion between expressions and improves user experience.

[0223] For an upcoming "clap" emoji, first determine its grid location, then check all adjacent grids (up, down, left, right) connected to the grid. If other high-weight emojis (such as "laughing") are detected in these adjacent grids, the system will consider them as potential conflicting emojis. Finally, based on the weight comparison, determine whether to filter out the conflicting emojis or just fine-tune their positions.

[0224] By adjusting the sensitivity of spatial indexing relationships and emoji display density, the system can intelligently optimize the layout of emoji data display, avoiding visual interference caused by excessive density of emojis. Not only does this improve the accuracy and aesthetics of emoji display, but it also reduces the computational complexity of spatial optimization, especially in real-time scenarios where emoji density changes dynamically. Through dynamic adjustment of grid size, the system can adapt to different scenarios, maintaining the cleanliness of the interface and the smoothness of user experience.

[0225] As an optional example, the above-mentioned filtering of potential conflicting emoji data according to the current filtering radius includes:

[0226] Obtain the emoji display density in the interface display area;

[0227] Determine the current filtering radius based on the emoji display density and the initial filtering radius;

[0228] Filter the potential conflicting emoji data according to the current filtering radius to obtain the second set of emoji data;

[0229] Wherein, the larger the current filtering radius, the more virtual emojis in the potential conflicting emojis are filtered out.

[0230] Specifically, referring to the above formula (10), first determine the emoji display density, then calculate the current filtering radius based on the default initial filtering radius R0, and filter the second set of emoji data according to the current filtering radius.

[0231] Wherein, based on the emoji display density, the filtering radius is dynamically adjusted to adapt to the current emoji distribution. The initial filtering radius is a system default setting, but will be adjusted according to the real-time changes in emoji density to achieve the best filtering effect.

[0232] According to the adjusted current filtering radius, the potential conflicting emojis are filtered, i.e. deciding which emoji data can be retained for display and which need to be adjusted or removed. This helps to reduce visual clutter on the UI interface and improve the clarity of emoji display and user experience.

[0233] For example, assume that in a region with a high density of expressions, the system determines that the current filtering radius is 1 cm. For each expression to be displayed, the system detects the expressions within a 1 cm radius around it and calculates the spatial overlap rate. If the spatial overlap rate between a "clapping" expression and its adjacent "laughing" expression exceeds a second threshold value (e.g., 0.5), the system determines whether to display the "clapping" expression or adjust its position to reduce occlusion based on the weights and priorities of the expressions.

[0234] After filtering, a second set of expression data is obtained, which contains the set of expressions that are most suitable for display under the current filtering conditions. The application of the filtering result not only optimizes expression display, but also reduces unnecessary rendering operations, reduces system resource consumption, and improves overall performance.

[0235] For example, the system applies the second set of expression data after filtering to update the UI interface, ensuring that each expression has sufficient display space, while adjusting the transparency or size of low-priority expressions to maintain visual appeal while avoiding performance bottlenecks, balancing visual effects and overall system performance.

[0236] The filtering radius is a key parameter of the enhanced spatial filtering algorithm, which determines the spatial range for judging whether expression data may conflict with other expressions in the spatial dimension. Specifically, the filtering radius refers to a circular region centered on the center point of each expression, and checks whether there are other expressions within this region. If one or more expressions have a distance from the center point of the current expression less than or equal to the filtering radius, it is determined that there is a spatial conflict between the two expressions.

[0237] The role of the filtering radius includes but is not limited to the following two points:

[0238] (1) Avoid overlapping and occlusion: By limiting the minimum display distance between expressions, it ensures that each expression has sufficient control to display and avoids poor visual effects caused by multiple expressions overlapping;

[0239] (2) Performance optimization: In the spatial filtering process, a larger filtering radius means that more expression data needs to be judged for conflict, increasing the computational complexity; while a smaller filtering radius can quickly exclude most irrelevant expression data, reducing the amount of calculation and improving filtering efficiency.

[0240] By dynamically adjusting the filtering radius, the filtering process of potential conflict expressions is optimized according to the real-time expression display density. It ensures that in the expression-intensive area, the system can reduce expression overlap with stricter filtering standards (increase the filtering radius) to avoid visual clutter, while in the sparse expression distribution area, the system relaxes the filtering conditions to allow more expressions to be displayed, improving the flexibility and compatibility of the technical solution.

[0241] As an optional example, the above-mentioned display of the target virtual expression corresponding to the second group of expression data according to the current spatial display parameter includes:

[0242] Based on the weight decay model, the comprehensive weight of the group of virtual expressions corresponding to the second group of expression data is determined, wherein the weight decay model is obtained based on a distance weight function between the group of virtual expressions, a time weight function based on the existence time of the expression data, and the initial priority of the expression data;

[0243] In the case where the comprehensive weight of each virtual expression in the group of virtual expressions meets the preset display condition, the target virtual expression is displayed according to the display position and transparency indicated by the current spatial display parameter;

[0244] In the case where the first virtual expression and the second virtual expression in the group of virtual expressions have spatial overlap, and the difference between the first comprehensive weight of the first virtual expression and the second comprehensive weight of the second virtual expression is less than a third threshold, the display position of the first virtual expression is adjusted or the first virtual expression and the second virtual expression are staggered displayed.

[0245] Referring to the weight decay model shown in the above formula (9), the comprehensive weight of each virtual expression in the second group of expression data is determined. The model considers three key factors: the distance between expressions, the existence time of expression data, and the initial priority. The distance weight function decreases with the increase of the distance between expressions, the time weight function decreases with the extension of the existence time of expression data, and the initial priority directly reflects the importance of the expression.

[0246] After calculating the comprehensive weights of the group of virtual expressions, it is checked whether these weights meet the preset display condition. If the condition is met, that is, the comprehensive weight reaches a certain threshold, the system will display the target virtual expression according to the current spatial display parameter (including display position and transparency), ensuring the visibility of high-weight expressions and user experience.

[0247] In addition, as Figure 6 shown, for the first virtual expression and the second virtual expression with spatial overlap, when the difference between the first comprehensive weight and the second comprehensive weight is less than a third threshold (for example, 0.1), one of the expressions will not be simply filtered out, but a more delicate processing strategy will be adopted.

[0248] Specifically, by fine-tuning the display positions of the expressions, making them staggered display, or using time staggered display method, the two expressions that may partially overlap are displayed alternately on the screen, thereby solving the problem of spatial overlap while ensuring the richness of expression display.

[0249] For example, when two "clap" expressions appear almost simultaneously at the same position on the screen, the system fine-tunes the offset of one of the expressions according to its combined weight, making it slightly move left or right, while reducing its transparency, thus preserving the continuity of emotional expression and avoiding visual conflict, and improving the user's visual experience.

[0250] In the case where the difference between the first combined weight of the first virtual expression and the second combined weight of the second virtual expression in a group of virtual expressions is less than a fourth threshold value (the fourth threshold value is less than the third threshold value), one of the expressions is not directly filtered out, but the transparency is reduced to reduce visual conflict.

[0251] In the above manner, by using the weight decay model, the intelligent management of expression display is realized through the calculation and analysis of the combined weight of the expression data. Not only the intuitive display of the expression is ensured, but also the time and space relationship between the expressions is fully considered. By dynamically adjusting the display position and transparency, the element overlap of the UI interface is effectively avoided. Especially in the case of serious spatial overlap, by fine-tuning or staggered display, the display opportunity of key expressions is ensured, while the risk of visual confusion is reduced, and the user's experience satisfaction in watching live broadcast, social interaction and other scenarios is significantly improved.

[0252] The technical solutions provided by the present application can at least solve the following technical problems:

[0253] (1) The problems of interface lag and UI overlap display in traditional expression data processing are solved, and the user experience smoothness is improved;

[0254] (2) The expression data processing capability of mobile terminal devices in weak network environment is optimized, and the data loss rate and display delay are reduced;

[0255] (3) Dynamic adjustment of batch processing and spatial filtering parameters is realized, which adapts to complex and variable application scenarios;

[0256] (4) The limitation of independent processing in time and space dimensions is broken through, and the global optimal allocation of system resources is realized.

[0257] By using the technical solutions in the above embodiments, the following beneficial effects can be achieved:

[0258] (1) Through the spatio-temporal optimization algorithm to achieve optimal allocation of computing resources, reduce the CPU and GPU peak load, reduce the data processing delay, reduce the memory occupation, while improving the interface smoothness;

[0259] (2) Significantly reduce the interface element overlap phenomenon, improve the system response speed and improve the continuity of expression display in weak network environment;

[0260] (3) Support more online users, reduce server bandwidth cost, reduce user loss due to performance problems, and improve business value.

[0261] It should be noted that for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0262] According to another aspect of the embodiments of the present application, a virtual expression display device is also provided, as shown in Figure 9 The device comprises:

[0263] A first acquisition unit 902 is configured to acquire a set of expression data to be processed, wherein the set of expression data is obtained according to real-time captured facial expressions in a real scene;

[0264] A first processing unit 904 is configured to determine a first group of expression data for current batch processing from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of batch processing.

[0265] A screening unit 906 is configured to screen the first group of expression data based on a current spatial screening parameter to obtain a second group of expression data to be displayed, wherein the spatial processing parameters include the current spatial screening parameter and a current spatial display parameter.

[0266] A first display unit 908 is configured to display a target virtual expression corresponding to the second group of expression data according to the current spatial display parameter.

[0267] Optionally, the first processing unit 904 comprises:

[0268] The first creation module is configured to create a comprehensive evaluation function based on a first evaluation function and a second evaluation function, wherein the first evaluation function is used to represent processing efficiency and real-time performance of batch processing in a time dimension, and the second evaluation function is used to represent visual effect coverage and spatial overlap rate of displaying virtual expressions in a space dimension.

[0269] The first acquisition module is configured to acquire a historical evaluation result of the comprehensive evaluation function after the last batch processing.

[0270] The update module is configured to update parameter configuration information in the comprehensive evaluation function based on the historical evaluation result, to obtain updated parameter configuration information, wherein the parameter configuration information includes a data processing frequency, a spatial processing parameter, and a preset weight coefficient.

[0271] The second acquisition module is configured to acquire a first group of expression data from the expression data set according to a time interval indicated by the current data processing frequency, wherein the updated parameter configuration information includes the current data processing frequency.

[0272] Optionally, the first creation module includes:

[0273] The first processing submodule is configured to perform weighted summation on the first evaluation function, the second evaluation function, and a space-time interaction term, to obtain the comprehensive evaluation function, wherein the space-time interaction term is used to capture a nonlinear interaction relationship of expression data in the expression data set in the time dimension and the space dimension.

[0274] Optionally, the update module includes:

[0275] The second processing submodule is configured to increase the data processing frequency and adjust a first group of weight coefficients in the first evaluation function in a case where the historical evaluation result indicates that data accumulation occurs in a batch processing queue, wherein the batch processing queue includes the expression data set stored in sequence according to the priority of the expression data.

[0276] The third processing submodule is configured to increase a screening radius and adjust a second group of weight coefficients in the second evaluation function in a case where the historical evaluation result indicates that the spatial overlap rate of the virtual expression reaches a preset threshold.

[0277] Optionally, the second acquisition module includes:

[0278] The first acquisition submodule is configured to acquire multi-modal load description information, wherein the multi-modal load description information includes network state data, device state data, and online interaction data.

[0279] The fourth processing submodule is configured to determine a current data amount of the current batch processing based on the network state data, the device state data, and the online interaction data.

[0280] The second obtaining sub-module is configured to obtain a first set of expression data from the expression data set according to a time interval and a current data volume.

[0281] Optionally, the screening unit 906 includes:

[0282] The first processing module is configured to determine an initial weight and an initial priority of each expression data in the first set of expression data.

[0283] The saving module is configured to save the first set of expression data to a plurality of priority queues pre-created according to the initial priority.

[0284] The extraction module is configured to extract expression data to be rendered from the plurality of priority queues based on the initial weight, wherein, in a case where the initial weight is greater than a first threshold value, a virtual expression corresponding to the expression data is allowed to be displayed.

[0285] The positioning module is configured to position potential conflict expressions having a spatial overlap rate greater than or equal to a second threshold value with a to-be-displayed virtual expression based on a spatial index relationship, wherein the to-be-displayed virtual expression is obtained based on the expression data to be rendered.

[0286] The first screening module is configured to screen expression data corresponding to the potential conflict expressions according to a current screening radius to obtain a second set of expression data, wherein the current spatial screening parameter includes the initial weight and the current screening radius.

[0287] Optionally, the positioning module includes:

[0288] The third obtaining sub-module is configured to obtain a spatial index relationship between a pre-created spatial position and an interface display area, wherein the spatial position is position information of a virtual agent obtained according to a scene layout in an actual scene.

[0289] The fourth obtaining sub-module is configured to obtain an expression display density in the interface display area.

[0290] The first adjusting sub-module is configured to adjust an initial grid size based on the expression display density to obtain an adjusted grid size, wherein the initial grid size is pre-set, and the interface display area is divided into a group of grids according to the initial grid size.

[0291] The fifth processing sub-module is configured to determine a current display area of the to-be-displayed virtual expression according to the spatial index relationship and the adjusted grid size.

[0292] The sixth processing sub-module is configured to determine adjacent grids connected with a grid occupied by the to-be-displayed virtual expression based on the current display area, and determine virtual expressions in the adjacent grids as the potential conflict expressions.

[0293] Optionally, the first screening module comprises:

[0294] The fifth obtaining sub-module is configured to obtain an expression display density in the interface display region.

[0295] The seventh processing sub-module is configured to determine a current screening radius based on the expression display density and the initial screening radius.

[0296] The screening sub-module is configured to screen the expression data corresponding to the potential conflict expression according to the current screening radius to obtain a second group of expression data; the greater the current screening radius, the more virtual expressions in the potential conflict expression are screened out.

[0297] Optionally, the first display unit 908 comprises:

[0298] The second processing module is configured to determine a comprehensive weight of a group of virtual expressions corresponding to the second group of expression data based on a weight decay model, wherein the weight decay model is obtained based on a distance weight function between the group of virtual expressions, a time weight function based on an existing time of the expression data, and an initial priority of the expression data.

[0299] The first display module is configured to display the target virtual expression according to a display position and a transparency indicated by the current spatial display parameter in a case where the comprehensive weight of each virtual expression in the group of virtual expressions meets a preset display condition.

[0300] The adjusting module is configured to adjust a display position of the first virtual expression or interleave the first virtual expression and the second virtual expression in a case where the first virtual expression and the second virtual expression in the group of virtual expressions have spatial overlap and a difference between the first comprehensive weight of the first virtual expression and the second comprehensive weight of the second virtual expression is less than a third threshold value.

[0301] It should be noted that the embodiments of the display device of the virtual expression can refer to the embodiments of the display method of the virtual expression described above, which will not be described here.

[0302] According to another aspect of the embodiments of the present application, an electronic device for implementing the display method of the virtual expression is also provided, which can be a target terminal or a server as shown. Figure 1 As shown in the figure, the electronic device comprises a memory 1002 and a processor 1004, the memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any of the method embodiments by the computer program. Figure 10

[0303] ​Optionally, in the embodiment, the electronic device can be located in at least one of the plurality of network devices of the computer network.

[0304] Optionally, the processor can be configured to perform the following steps by means of a computer program:

[0305] S1, obtaining a set of expression data to be processed, wherein the set of expression data is obtained according to real-time captured facial expressions in a real scene;

[0306] S2, determining a first group of expression data for current batch processing from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of batch processing;

[0307] S3, screening the first group of expression data based on the current spatial screening parameter to obtain a second group of expression data to be displayed, wherein the spatial processing parameter includes the current spatial screening parameter and the current spatial display parameter;

[0308] S4, displaying a target virtual expression corresponding to the second group of expression data according to the current spatial display parameter.

[0309] Optionally, those skilled in the art can understand that, Figure 10 The structure shown is only schematic, Figure 10 which does not limit the structure of the electronic device. Figure 10 For example, the electronic device can include more or less components than those shown, or have a different configuration of components than those shown. Figure 10 For example, the electronic device can include more or less components than those shown, or have a different configuration of components than those shown.

[0310] The memory 1002 can be used to store software programs and modules, such as program instructions / modules corresponding to the display method and device of virtual expression in the embodiments of the present application, and the processor 1004 performs various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, implements the display method of virtual expression described above. The memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 1002 can further include a memory remotely arranged with respect to the processor 1004, which can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. Specifically, the memory 1002 can be but is not limited to being used to store key video frames, multi-modal search information, and historical behavior data, etc. As an example, the memory 1002 can be used to store the key video frames, multi-modal search information, and historical behavior data, etc. as shown in FIG. 1. Figure 10As shown, the memory 1002 can include, but is not limited to, the first acquisition unit 902, the first processing unit 904, the screening unit 906, and the first display unit 908 in the virtual expression display device. In addition, other module units in the virtual expression display device can also be included, which will not be described herein.

[0311] Optionally, the transmission device 1006 is configured to receive or send data via a network. The network can include a wired network and a wireless network. In an example, the transmission device 1006 includes a network adapter (NIC) that can be connected to other network devices and routers through a network cable to communicate with the Internet or a local area network. In an example, the transmission device 1006 is a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0312] In addition, the electronic device further includes a display 10010 configured to display a video frame of a target video, and a connection bus 1010 configured to connect various module components in the electronic device.

[0313] In other embodiments, the target terminal or server can be a node in a distributed system, and the distributed system can be a blockchain system formed by the plurality of nodes connected through network communication. The nodes can form a point-to-point network, and any computing device, such as a server, a target terminal, or an electronic device, can become a node in the blockchain system by joining the point-to-point network.

[0314] According to another aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the virtual expression display method provided in various optional implementation manners of the server verification processing aspect, wherein the computer program is configured to execute the steps in any of the method embodiments when running.

[0315] Optionally, in the present embodiment, the computer readable storage medium can be configured to store a computer program for executing the following steps:

[0316] S1, acquiring a set of expression data to be processed, wherein the set of expression data is obtained according to real-time captured facial expressions in a real scene;

[0317] S2, determining a first group of expression data of the current batch from the set of expression data based on a current data processing frequency indicated by an evaluation result of a comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and the spatial processing parameter of the batch processing;

[0318] S3, performing screening on the first group of expression data based on the current spatial screening parameter to obtain a second group of expression data to be displayed, wherein the spatial processing parameter comprises the current spatial screening parameter and a current spatial display parameter;

[0319] S4, displaying a target virtual expression corresponding to the second group of expression data according to the current spatial display parameter.

[0320] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0321] Optionally, in the embodiments, a person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the hardware related to the target terminal by a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0322] The serial numbers of the above embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0323] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the embodiments of the present application.

[0324] In the above-described embodiments of the present application, the description of each embodiment is focused on each aspect, and the part not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0325] In several embodiments provided in the present application, it should be understood that the disclosed client can be implemented by other manners. Among them, the above-described device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, and can be in electrical or other forms.

[0326] The unit described as a separate component can be or can not be physically separated, and the component displayed as a unit can be or can not be a physical unit, that is, it can be located in one place, or it can be distributed to a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0327] In addition, each functional unit in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0328] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for displaying virtual facial expressions, characterized in that, include: Obtain a set of facial expression data to be processed, wherein the set of facial expression data is obtained based on real-time captured facial expressions in a real scene; Based on the evaluation result indicated by the comprehensive evaluation function, the first set of facial expression data for the current batch processing is determined from the facial expression data set. The comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of the batch processing. Based on the current spatial filtering parameters, the first set of expression data is filtered to obtain the second set of expression data to be displayed. The spatial processing parameters include the current spatial filtering parameters and the current spatial display parameters. Display the target virtual expression corresponding to the second set of expression data according to the current spatial display parameters.

2. The method according to claim 1, characterized in that, The current data processing frequency indicated by the evaluation result based on the comprehensive evaluation function determines the first set of facial expression data for the current batch processing from the facial expression data set, including: Based on the first evaluation function and the second evaluation function, the comprehensive evaluation function is created, wherein the first evaluation function is used to characterize the processing efficiency and real-time performance of batch processing in the time dimension, and the second evaluation function is used to characterize the visual effect coverage and spatial overlap rate of displaying virtual expressions in the spatial dimension. Retrieve the historical evaluation results of the comprehensive evaluation function since the last batch processing; Based on the historical evaluation results, the parameter configuration information in the comprehensive evaluation function is updated to obtain the updated parameter configuration information, wherein the parameter configuration information includes the data processing frequency, the spatial processing parameters, and the preset weight coefficients; According to the time interval indicated by the current data processing frequency, the first set of facial expression data is obtained from the facial expression data set, wherein the updated parameter configuration information includes the current data processing frequency.

3. The method according to claim 2, characterized in that, The creation of the comprehensive evaluation function based on the first evaluation function and the second evaluation function includes: The first evaluation function, the second evaluation function, and the spatiotemporal interaction term are weighted and summed to obtain the comprehensive evaluation function, wherein the spatiotemporal interaction term is used to capture the nonlinear interaction relationship between the facial expression data in the facial expression dataset in the time dimension and the spatial dimension.

4. The method according to claim 2, characterized in that, The step of updating the parameter configuration information in the comprehensive evaluation function based on the historical evaluation results to obtain the updated parameter configuration information includes: When the historical evaluation results indicate that there is a data backlog in the batch processing queue, the data processing frequency is increased and the first set of weight coefficients in the first evaluation function is adjusted, wherein the batch processing queue includes the set of facial expression data stored sequentially according to the priority of the facial expression data; If the historical evaluation results indicate that the spatial overlap rate of the virtual expression has reached a preset threshold, the filtering radius is increased and the second set of weight coefficients in the second evaluation function is adjusted.

5. The method according to claim 2, characterized in that, The step of obtaining the first set of facial expression data from the facial expression data set according to the time interval indicated by the current data processing frequency includes: Obtain multimodal load description information, wherein the multimodal load description information includes network status data, device status data, and online interaction data; Based on the network status data, the device status data, and the online interaction data, the current data volume of the current batch processing is determined; The first set of facial expression data is obtained from the facial expression data set according to the time interval and the current data volume.

6. The method according to claim 1, characterized in that, The first set of expression data is filtered based on the current spatial filtering parameters to obtain the second set of expression data to be displayed, including: Determine the initial weight and initial priority of each facial expression data in the first set of facial expression data; According to the initial priority, the first set of facial expression data is saved to multiple pre-created priority queues; Based on the initial weight, the expression data to be rendered is extracted from the multiple priority queues, wherein if the initial weight is greater than a first threshold, the display of virtual expressions corresponding to the expression data is allowed; Based on spatial indexing relationships, potential conflicting expressions with a spatial overlap rate greater than or equal to a second threshold are located, wherein the virtual expressions to be displayed are obtained based on the expression data to be rendered; According to the current filtering radius, the expression data corresponding to the potential conflict expressions are filtered to obtain the second set of expression data, wherein the current spatial filtering parameters include the initial weight and the current filtering radius.

7. The method according to claim 6, characterized in that, The method of locating potential conflicting expressions based on spatial indexing relationships, where the spatial overlap rate between the expression to be displayed and the virtual expression is greater than or equal to a second threshold, includes: Obtain the spatial index relationship between the pre-created spatial location and the interface display area, wherein the spatial location is the location information of the virtual seat obtained based on the scene layout in the actual scene; Obtain the facial expression display density in the interface display area; Based on the expression display density, the initial grid size is adjusted to obtain the adjusted grid size. The initial grid size is preset, and the interface display area is divided into a group of grids according to the initial grid size. Based on the spatial index relationship and the adjusted grid size, determine the current display area of ​​the virtual emoticon to be displayed; Based on the current display area, neighboring grids connected to the grid occupied by the virtual emoticon to be displayed are determined, and the virtual emoticons in the neighboring grids are identified as the potential conflict emoticons.

8. The method according to claim 6, characterized in that, The process involves filtering the expression data corresponding to the potentially conflicting expressions according to the current filtering radius to obtain the second set of expression data, including: Get the facial expression display density in the interface display area; The current filtering radius is determined based on the expression display density and the initial filtering radius; Based on the current filtering radius, the expression data corresponding to the potential conflicting expressions are filtered to obtain the second set of expression data; The larger the current filtering radius, the more virtual expressions are filtered out from the potential conflict expressions.

9. The method according to any one of claims 1 to 8, characterized in that, The step of displaying the target virtual expression corresponding to the second set of expression data according to the current spatial display parameters includes: Based on the weight decay model, the comprehensive weight of a set of virtual expressions corresponding to the second set of expression data is determined. The weight decay model is determined by a distance weight function based on the distance between the set of virtual expressions, a time weight function based on the existence time of the expression data, and the initial priority of the expression data. When the overall weight of each virtual expression in the set of virtual expressions meets the preset display conditions, the target virtual expression is displayed according to the display position and transparency indicated by the current spatial display parameters. If the first virtual expression and the second virtual expression in the set of virtual expressions have spatial overlap, and the difference between the first comprehensive weight of the first virtual expression and the second comprehensive weight of the second virtual expression is less than the third threshold, the display position of the first virtual expression is adjusted or the first virtual expression and the second virtual expression are displayed alternately.

10. A display device for virtual facial expressions, characterized in that, include: The first acquisition unit is used to acquire a set of facial expression data to be processed, wherein the set of facial expression data is obtained based on facial expressions captured in real time in a real scene; The first processing unit is used to determine the first set of facial expression data for the current batch processing from the facial expression data set based on the current data processing frequency indicated by the evaluation result of the comprehensive evaluation function, wherein the comprehensive evaluation function is used to dynamically adjust the data processing frequency and spatial processing parameters of the batch processing. A filtering unit is used to filter the first set of expression data based on the current spatial filtering parameters to obtain the second set of expression data to be displayed, wherein the spatial processing parameters include the current spatial filtering parameters and the current spatial display parameters; The first display unit is used to display the target virtual expression corresponding to the second set of expression data according to the current spatial display parameters.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or computer at runtime as described in any one of claims 1 to 9.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to perform the method as described in any one of claims 1 to 9 via the computer program.

Images