Method and device for processing advertisement delivery in virtual world

By collecting user EEG signals and eye images in the virtual world, combining emotion detection models and virtual environment data, the problem of poor advertising delivery in the virtual world is solved, and more accurate and efficient advertising delivery is achieved.

CN115358777BActive Publication Date: 2024-12-06ANT DIGITAL INFORMATION TECHNOLOGY (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210983372.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-16
Publication Date
2024-12-06
Estimated Expiration
2042-08-16

AI Technical Summary

Technical Problem

In the virtual world, existing advertising delivery technology is difficult to accurately detect the user's emotional state and the matching degree of virtual environment, resulting in poor advertising delivery and low conversion rate.

Method used

By collecting the user's EEG signals and eye images, inputting the emotion detection model for emotion detection, obtaining multimodal emotional characteristics and emotion levels, combining virtual environment data to calculate the matching degree between the advertisement and the environment, and determining the advertising delivery.

Benefits of technology

It has achieved more accurate and targeted advertising delivery in the virtual world, and improved advertising delivery performance and conversion rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115358777B_ABST
    Figure CN115358777B_ABST
Patent Text Reader

Abstract

The embodiments of the present specification provide a method and device for processing advertisement delivery in a virtual world, wherein a method for processing advertisement delivery in a virtual world comprises: inputting a user's electroencephalogram signal and eye image into an emotion detection model for emotion detection processing, obtaining multimodal emotion features and emotion levels of at least one emotion dimension; performing advertisement delivery detection in the virtual world based on the multimodal emotion features and the emotion levels; if the detection passes, calculating the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world; and determining an advertisement to be delivered from the candidate advertisement based on the matching degree and performing delivery processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This document relates to the field of virtualization technology, and in particular to a method and device for processing advertisement delivery in a virtual world. Background Art

[0002] The virtual world provides a simulation of the real world, and can even provide scenes that are difficult to achieve in the real world. Therefore, the virtual world is increasingly used in various scenarios. In the virtual world scenario, users log in to the three-dimensional virtual world through a specific identity ID and carry out activities through virtual user roles in the virtual world. Usually, there are different user roles in the virtual world, and these user roles each carry out different activities. Summary of the invention

[0003] One or more embodiments of the present specification provide a method for processing advertisement delivery in a virtual world, including: inputting a user's electroencephalogram signal and eye image into an emotion detection model for emotion detection processing, and obtaining multimodal emotion features and emotion levels of at least one emotion dimension. Based on the multimodal emotion features and the emotion levels, a virtual world advertisement delivery detection is performed. If the detection passes, a matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world is calculated. Based on the matching degree, an advertisement is determined to be delivered from the candidate advertisements and the delivery process is performed.

[0004] One or more embodiments of the present specification provide an advertising delivery processing device for a virtual world, including: an emotion detection processing module, configured to input a user's electroencephalogram signal and eye image into an emotion detection model for emotion detection processing, and obtain multimodal emotion features and emotion levels of at least one emotion dimension. An advertising delivery detection module, configured to perform advertising delivery detection in the virtual world based on the multimodal emotion features and the emotion levels. A matching degree calculation module, configured to calculate the degree of match between the advertising data of a candidate advertisement and the virtual environment data of the virtual world if the detection passes. A delivery processing module, configured to determine an advertisement to be delivered from the candidate advertisements based on the matching degree and perform delivery processing.

[0005] One or more embodiments of the present specification provide an advertising delivery processing device for a virtual world, comprising: a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor: inputs the user's electroencephalogram signals and eye images into an emotion detection model for emotion detection processing, and obtains multimodal emotion features and emotion levels of at least one emotion dimension. Based on the multimodal emotion features and the emotion levels, the advertising delivery detection of the virtual world is performed. If the detection passes, the matching degree between the advertising data of the candidate advertisement and the virtual environment data of the virtual world is calculated. Based on the matching degree, the advertisement to be delivered is determined from the candidate advertisements and the delivery processing is performed.

[0006] One or more embodiments of the present specification provide a storage medium for storing computer executable instructions, which implement the following process when executed by a processor: input the user's electroencephalogram signal and eye image into an emotion detection model for emotion detection processing, and obtain multimodal emotion features and emotion levels of at least one emotion dimension. Perform advertising delivery detection in the virtual world based on the multimodal emotion features and the emotion levels. If the detection passes, calculate the matching degree between the advertising data of the candidate advertisement and the virtual environment data of the virtual world. Determine the advertisement to be delivered from the candidate advertisement based on the matching degree and perform delivery processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0008] Figure 1 A processing flow chart of a method for processing advertisement delivery in a virtual world provided by one or more embodiments of this specification;

[0009] Figure 2 A processing flow chart of a method for processing advertisement delivery in a virtual world applied to a virtual asset scenario provided by one or more embodiments of this specification;

[0010] Figure 3 A schematic diagram of an advertising delivery processing device for a virtual world provided by one or more embodiments of this specification;

[0011] Figure 4 A schematic diagram of the structure of a virtual world advertising delivery processing device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0013] This specification provides an embodiment of a method for processing advertisement delivery in a virtual world:

[0014] The present application provides a method for processing advertisement delivery in a virtual world, which performs emotion detection processing by inputting the collected EEG information and eye images of the user into an emotion detection model to obtain multimodal emotion features and emotion levels of at least one emotion dimension, and then performs advertisement delivery detection on the virtual world accessed by the user based on the multimodal emotion features and emotion levels. If the detection passes, the degree of match between the candidate advertisement and the current virtual environment data of the virtual world is further calculated, and finally, the advertisement to be delivered is determined based on the degree of match and delivery processing is performed, thereby performing advertisement delivery detection and matching in the virtual world from multiple modalities, thereby performing more accurate and targeted advertisement delivery in the virtual world, and further improving the conversion effect of advertisement delivery in the virtual world on the basis of improving the advertisement delivery effect in the virtual world.

[0015] Reference Figure 1 The present embodiment provides a method for processing advertisement delivery in a virtual world, and the method specifically includes steps S102 to S108.

[0016] Step S102: Input the user's EEG signal and eye image into an emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension.

[0017] The EEG signal described in this embodiment refers to the electrical signal of the user's brain activity collected by the EEG sensor. The eye image refers to an image containing the user's eye features collected by the image sensor. Optionally, in the scenario where the user accesses the virtual world, the EEG signal is collected by a signal sensor integrated or configured in the access device of the virtual world, such as collecting the user's EEG signal through the EEG sensor configured in the access device; the eye image is collected by an image sensor integrated or configured in the access device of the virtual world. This embodiment enhances the comprehensiveness of the data collected when the user accesses the virtual world through the access device by collecting data in two modes, namely, EEG signals and eye images, and thereby improves the accuracy of emotion detection of the user visiting the virtual world through multimodal data.

[0018] The virtual world refers to a virtual simulation world based on decentralized collaboration and with an open economic system, such as a game virtual world in which users interact with each other through access devices, a conference virtual world in which users conduct online meetings through access devices, or a transaction virtual world in which users conduct online transactions through access devices; optionally, decentralized transactions are conducted in the virtual world by generating non-homogeneous identifiers, and ownership of virtual assets is obtained through transactions; in this embodiment, during the process of advertising delivery when a user accesses the virtual world through an access device, the candidate advertisements delivered in the virtual world include asset advertisements of virtual assets corresponding to the non-homogeneous identifiers. The access device of the virtual world may be a VR (Virtual Reality) device, an AR (Augmented Reality) device, etc., connected to the virtual world, such as a head-mounted VR device connected to the virtual world.

[0019] In a specific implementation, in the process of detecting the user's emotions based on the EEG signal and the eye image, the EEG signal and the eye image are input into an emotion detection model to realize the detection of the user's emotions based on multimodal data (EEG signal and eye image). Optionally, the emotion detection model includes: a first network, a second network and a fusion network; wherein the first network is used to perform feature extraction on the EEG signal to obtain EEG signal features; the second network is used to perform feature extraction on the eye image to obtain eye image features; the fusion network is used to perform feature fusion on the EEG signal features and the eye image features, and to perform emotion detection on the user based on the EEG signal features and the eye image features to obtain an emotion level.

[0020] On the basis that the emotion detection model includes the first network, the second network and the fusion network, in an optional implementation manner provided by this embodiment, the emotion detection model specifically performs emotion detection processing in the following manner:

[0021] Inputting the EEG signal into the first network to perform feature extraction to obtain EEG signal features, and inputting the eye image into the second network to perform feature extraction to obtain eye image features;

[0022] The EEG signal features and the eye image features are input into the fusion network for feature processing to obtain the multimodal emotion features and the emotion level.

[0023] Furthermore, in an optional implementation manner provided by this embodiment, the feature processing performed by the fusion network includes:

[0024] Performing feature fusion on the EEG signal feature and the eye image feature, and using the obtained fusion feature as the multimodal emotion feature;

[0025] Emotion detection of the at least one emotion dimension is performed based on the EEG signal features and the eye image features to obtain an emotion level of the at least one emotion dimension.

[0026] The emotion dimension includes: attention dimension, pleasure dimension and / or tension dimension; the attention dimension refers to a dimension used to characterize the user's attention to the accessed content in the virtual world, the pleasure dimension refers to a dimension used to characterize the user's pleasure in the process of accessing the virtual world, and the tension dimension refers to a dimension used to characterize the tension of the user in the process of accessing the virtual world.

[0027] In practical applications, the training of the emotion detection model can be completed in advance, such as performing model training of the emotion detection model on a cloud server. Specifically, during the training process of the emotion detection model, a loss function can also be constructed, such as constructing a loss function based on visual emotion loss, EEG emotion loss, and multimodal emotion loss. During the training process, the network is trained based on the above-mentioned network structure and loss function until the emotion detection model is obtained after the network converges.

[0028] In addition, the above-mentioned input of the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and emotion levels of at least one emotion dimension can also be replaced by performing feature fusion on the EEG signal features and the eye image features to obtain multimodal emotion features, and determining the emotion level of at least one emotion dimension based on the EEG signal features and the eye image features, extracting features from the user's EEG signal using the EEG signal features, and extracting features from the user's eye image using the eye image features, and forming a new implementation method together with the following steps provided in this embodiment.

[0029] Step S104: performing advertisement placement detection in the virtual world based on the multimodal emotion feature and the emotion level.

[0030] After obtaining the multimodal characteristic emotion feature and the emotion level of at least one emotion dimension, an advertisement delivery detection is performed in the virtual world based on the multimodal emotion feature and the emotion level, so as to detect whether advertisement delivery is performed in the virtual world from the perspective of the emotion of the user currently visiting the virtual world.

[0031] In a specific implementation, during the process of detecting the advertisement delivery in the virtual world, in order to protect the user's access experience of the virtual world to the greatest extent, thereby improving the advertisement delivery effect in the virtual world as much as possible under the premise of ensuring the user's access experience, in an optional implementation manner provided by this embodiment, the advertisement delivery detection in the virtual world is performed based on the multimodal emotion feature and the emotion level, including:

[0032] Detecting whether the emotion level of each emotion dimension is greater than or equal to the emotion level threshold of the corresponding emotion dimension, and whether the advertisement delivery probability determined based on the multimodal emotion feature is greater than a preset probability threshold, and if so, determining that the detection result is a pass;

[0033] If not, it is determined that the test result is failed and no processing is performed.

[0034] Optionally, the advertisement delivery probability is obtained by inputting the multimodal emotion feature into a delivery detection model to perform advertisement delivery adaptation calculation; wherein, the advertisement delivery adaptation calculation includes calculating the fitness for advertisement delivery at the current time point based on the multimodal emotion feature as the advertisement delivery probability.

[0035] In addition, the advertising delivery detection in the virtual world based on the multimodal emotional features and the emotional level can also be achieved in the following way: detecting whether the multimodal emotional features and the emotional level meet the advertising delivery conditions, and if so, determining the detection result as a passed detection, and if not, determining the detection result as a failed detection; wherein the advertising delivery conditions include: the emotional level of each emotional dimension is greater than or equal to the emotional level threshold of the corresponding emotional dimension, and / or, the advertising delivery probability determined based on the multimodal emotional features is greater than a preset probability threshold, and the advertising delivery probability can be obtained by inputting the multimodal emotional features into the delivery detection model and performing an advertising delivery adaptation calculation.

[0036] In addition, the above-mentioned inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and emotion levels of at least one emotion dimension can also be replaced by extracting features from the user's EEG signal and eye image, and performing feature fusion on the EEG signal features and eye image features obtained by feature extraction to obtain multimodal emotion features. Accordingly, the above-mentioned detection of advertising placement in the virtual world based on the multimodal emotion features and the emotion levels can also be replaced by detection of advertising placement in the virtual world based on the multimodal emotion features, and together with the following steps provided in this embodiment, constitute a new implementation method;

[0037] Alternatively, the above-mentioned input of the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and emotion levels of at least one emotion dimension can also be replaced by inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain emotion levels of at least one emotion dimension. Correspondingly, the above-mentioned detection of advertising delivery in the virtual world based on the multimodal emotion features and the emotion levels can also be replaced by detection of advertising delivery in the virtual world based on the emotion levels of at least one emotion dimension, and together with the following steps provided in this embodiment, a new implementation method is formed. Among them, the specific implementation process of detecting advertising delivery in the virtual world based on multimodal emotion features or detecting advertising delivery in the virtual world based on the emotion levels of at least one emotion dimension can be referred to the implementation method provided above, which will not be repeated here.

[0038] Step S106: If the detection is successful, the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world is calculated.

[0039] In the case where the detection result of the above-mentioned detection of advertising placement in the virtual world based on the multimodal emotional features and the emotional level is that the detection has passed, starting from the advertising data of the candidate advertisements and the virtual environment data of the virtual world, the matching degree of the advertising data of each candidate advertisement and the virtual environment data of the virtual world is calculated respectively, that is, the matching degree between the candidate advertisement and the current virtual environment of the virtual world is calculated, and this is used as the basis for determining the placement of advertisements in the candidate advertisements in the subsequent process. Among them, the virtual environment data refers to the data of the virtual environment (for example, the virtual asset trading environment, the virtual social environment, the virtual game environment) or the virtual scene in the virtual world, and the virtual environment or the virtual scene can be a three-dimensional space constructed based on images in the virtual world. In the specific execution process, the virtual environment data can be the virtual environment data of the virtual environment or virtual scene currently displayed in the virtual world, or it can be the virtual environment data of the virtual environment or virtual scene between the current advertising placement point and the previous advertising placement point.

[0040] In a specific implementation, in the process of calculating the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world, in order to improve the calculation efficiency of the matching degree between the advertisement data and the virtual environment data, an optional implementation provided by this embodiment is to extract features of the key image contained in the advertisement data and the environment image contained in the virtual environment data to obtain advertisement key features and virtual environment features; and calculate the feature similarity between the advertisement key features and the virtual environment features as the matching degree. For example, the cosine similarity between the advertisement key features and the virtual environment features is calculated as the matching degree.

[0041] In actual applications, candidate advertisements may involve advertisements of different assets, which may cause the advertising data of the candidate advertisements to be more complicated. At the same time, the data type of the virtual environment data may also involve data of different virtual environments or virtual scenes. Therefore, in order to improve the efficiency and accuracy of feature extraction of the advertising data and virtual environment data of the candidate advertisements, feature extraction can be performed on the advertising data and virtual environment data of the candidate advertisements by training a feature extraction model. Specifically, in an optional implementation manner provided by this embodiment, feature extraction is performed on the key image contained in the advertising data and the environment image contained in the virtual environment data to obtain advertising key features and virtual environment features, including: inputting the key image into the feature extraction model for feature extraction to obtain the advertising key features, and inputting the environment image into the feature extraction model for feature extraction to obtain the virtual environment features.

[0042] As described above, the advertisement data of the candidate advertisements may be relatively complex, and the data type of the virtual environment data may also involve data of different virtual environments or virtual scenes. Therefore, in the process of training the feature extraction model for extracting features from the advertisement data and virtual environment data of the candidate advertisements, a large number of training samples need to be collected for model training. In view of this, in order to improve the training efficiency of the feature model, and at the same time to reduce the difficulty of collecting training samples and the workload of model training, in an optional implementation manner provided in this embodiment, model training is performed on the basis of constructing positive / negative sample pairs. In an optional implementation manner provided in this embodiment, the feature extraction model is trained and obtained in the following manner:

[0043] Inputting the positive sample pairs and / or negative sample pairs in the training sample set into the residual network to be trained for feature extraction to extract features, and calculating the feature similarity of the output sample features; wherein the residual network may adopt a ResNet50 or ResNet model architecture;

[0044] The training loss is calculated according to the feature similarity, and the parameters of the residual network are adjusted based on the training loss, and the feature extraction model is obtained after the training is completed.

[0045] Optionally, the positive sample pair includes a virtual environment image pair obtained by sampling from an image set of the same virtual environment; and the negative sample pair includes a virtual environment image pair obtained by sampling from an image set of different virtual environments.

[0046] In the specific implementation process, model training requires a large number of training samples, and it may be difficult to collect a large number of virtual environment image sets from the virtual world. There may not be enough virtual environment image sets to meet the model training. In view of this, this embodiment provides an optional implementation method, starting from the image of the physical environment (real environment), image creation and reconstruction are performed to obtain an image set of the virtual environment in the virtual world. Specifically, the image set of the virtual environment is generated in the following manner:

[0047] The physical environment image obtained by multi-view image acquisition of the physical environment is input into a pre-trained neural radiation field model for three-dimensional image reconstruction, and the output image corresponding to the three-dimensional virtual environment in the virtual world is used as the image set of the virtual environment. The neural radiation field model can adopt the NeRF (Neural Radiance Fields) model.

[0048] In practical applications, the training of the feature extraction model can be completed in advance, such as performing model training of the feature extraction model on a cloud server. Specifically, during the training process of the feature extraction model, a loss function can also be constructed. The loss function is specifically:

[0049]

[0050] Among them, f p1 、f p2 is the feature value for feature extraction of two training samples in the positive sample pair, f n1 、f n2 is the eigenvalue for feature extraction of two training samples in the negative sample pair, m is a large preset constant, ||f p1 -f p2 || 2 Indicates f p1 -f p2 Find the binorm, or f p1 With f p2 The Euclidean distance of Indicates f p1 -f p2 Square the result obtained by taking the two-norm, or f p1 With f p2 The square of the Euclidean distance. Through this loss function, the distance of the features extracted for the positive sample pairs in the feature space can be minimized, and the distance of the features extracted for the negative sample pairs in the feature space can be close to m. During the training process, the network is trained based on the above network structure and loss function until the network converges to obtain a feature extraction model.

[0051] In addition, in order to further improve the efficiency and accuracy of feature extraction of advertising data and virtual environment data of candidate advertisements, it is also possible to train an advertising feature extraction model for feature extraction of advertising data of candidate advertisements, and train a virtual environment feature extraction model for feature extraction of virtual environment data. By performing feature extraction through more targeted models, the accuracy of feature extraction, the training difficulty of the feature extraction model and the amount of computation during the training process can be improved.

[0052] Step S108: determining an advertisement to be delivered from the candidate advertisements based on the matching degree and performing delivery processing.

[0053] Based on the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world calculated in the above step, in this step, the advertisement to be delivered in the virtual world is determined from the candidate advertisement based on the matching degree, and delivery processing is performed on the determined advertisement.

[0054] In an optional implementation manner provided by this embodiment, determining a delivery advertisement from the candidate advertisements based on the matching degree and performing delivery processing includes:

[0055] Sorting the candidate advertisements in descending order of the matching degree, and selecting the candidate advertisements ranked in the top N positions; wherein N is a positive integer;

[0056] A candidate advertisement is randomly selected from the first N candidate advertisements as the delivered advertisement, and the delivered advertisement is rendered and output.

[0057] Here, rendering and outputting the delivered advertisement refers to rendering an advertisement display page of the delivered advertisement in the virtual world, and outputting the rendered advertisement display page to the access device of the user for display.

[0058] The following takes the application of a virtual world advertising processing method provided by this embodiment in a virtual asset scenario as an example. Figure 2 , further describing the method for processing advertisement delivery in the virtual world provided in this embodiment, see Figure 2 , an advertising delivery processing method for a virtual world applied to a virtual asset scenario specifically includes the following steps.

[0059] Step S202, obtaining an electroencephalogram signal collected by a signal sensor integrated in an access device for the user to access the virtual world, and an eye image collected by an image sensor integrated in the access device.

[0060] Step S204: input the EEG signal into the first network of the emotion detection model to perform feature extraction to obtain EEG signal features, and input the eye image into the second network of the emotion detection model to perform feature extraction to obtain eye image features.

[0061] Step S206, inputting the EEG signal features and the eye image features into the fusion network of the emotion detection model for feature processing to obtain multimodal emotion features and emotion levels.

[0062] Among them, the feature processing performed by the fusion network includes:

[0063] Perform feature fusion on EEG signal features and eye image features, and use the obtained fusion features as multimodal emotion features;

[0064] Emotion detection of at least one emotion dimension is performed based on the EEG signal features and the eye image features to obtain an emotion level of at least one emotion dimension.

[0065] Step S208, detecting whether the emotion level of each emotion dimension is greater than or equal to the emotion level threshold of the corresponding emotion dimension, and whether the advertisement delivery probability determined based on the multimodal emotion feature is greater than a preset probability threshold;

[0066] If yes, determine that the test result is passed, and execute the following steps S210 to S216;

[0067] If not, if not, it is determined that the test result is failed and no processing is performed.

[0068] Step S210, inputting the key image of the asset advertisement of the virtual asset into the feature extraction model for feature extraction to obtain the key features of the advertisement, and inputting the environment image of the current virtual environment in the virtual world into the feature extraction model for feature extraction to obtain the virtual environment features.

[0069] Step S212, calculating the feature similarity between the key features of the advertisement and the features of the virtual environment.

[0070] Step S214, sorting the asset advertisements in descending order of feature similarity, and selecting the asset advertisements with the top N rankings;

[0071] Step S216: randomly select an asset advertisement from the first N asset advertisements as a delivery advertisement, and render and output the delivery advertisement.

[0072] An embodiment of a virtual world advertising processing device provided in this specification is as follows:

[0073] In the above-mentioned embodiment, a method for processing advertisement delivery in a virtual world is provided, and correspondingly, a device for processing advertisement delivery in a virtual world is also provided, which will be described below with reference to the accompanying drawings.

[0074] Reference Figure 3 , which shows a schematic diagram of an advertising delivery processing device for a virtual world provided by this embodiment.

[0075] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0076] This embodiment provides an advertisement delivery processing device for a virtual world, including:

[0077] The emotion detection processing module 302 is configured to input the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension;

[0078] An advertisement delivery detection module 304 is configured to perform advertisement delivery detection in the virtual world based on the multimodal emotion feature and the emotion level;

[0079] A matching degree calculation module 306 is configured to calculate the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world if the detection is passed;

[0080] The delivery processing module 308 is configured to determine a delivery advertisement from the candidate advertisements based on the matching degree and perform delivery processing.

[0081] An embodiment of an advertisement delivery processing device for a virtual world provided in this specification is as follows:

[0082] Corresponding to the above-described method for processing advertisement delivery in a virtual world, based on the same technical concept, one or more embodiments of this specification further provide a device for processing advertisement delivery in a virtual world, and the device for processing advertisement delivery in a virtual world is used to execute the above-described method for processing advertisement delivery in a virtual world. Figure 4 A schematic diagram of the structure of a virtual world advertising delivery processing device provided in one or more embodiments of this specification.

[0083] This embodiment provides a virtual world advertising processing device, including:

[0084] like Figure 4As shown, the advertising delivery processing device of the virtual world may have relatively large differences due to different configurations or performances, and may include one or more processors 401 and memory 402, and the memory 402 may store one or more storage applications or data. Among them, the memory 402 may be a short-term storage or a persistent storage. The application stored in the memory 402 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the advertising delivery processing device of the virtual world. Furthermore, the processor 401 may be configured to communicate with the memory 402, and execute a series of computer executable instructions in the memory 402 on the advertising delivery processing device of the virtual world. The advertising delivery processing device of the virtual world may also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input / output interfaces 405, one or more keyboards 406, etc.

[0085] In a specific embodiment, the advertising delivery processing device of the virtual world includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the advertising delivery processing device of the virtual world, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following:

[0086] Inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension;

[0087] Performing advertisement placement detection in a virtual world based on the multimodal emotion features and the emotion levels;

[0088] If the detection is passed, calculating the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world;

[0089] Based on the matching degree, a delivery advertisement is determined from among the candidate advertisements and delivery processing is performed.

[0090] An embodiment of a storage medium provided in this specification is as follows:

[0091] Corresponding to the above-described method for processing advertisement delivery in a virtual world, based on the same technical concept, one or more embodiments of this specification further provide a storage medium.

[0092] The storage medium provided in this embodiment is used to store computer executable instructions, and the computer executable instructions implement the following process when executed by a processor:

[0093] Inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension;

[0094] Performing advertisement placement detection in a virtual world based on the multimodal emotion features and the emotion levels;

[0095] If the detection is passed, calculating the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world;

[0096] Based on the matching degree, a delivery advertisement is determined from among the candidate advertisements and delivery processing is performed.

[0097] It should be noted that the embodiment of the storage medium in this specification and the embodiment of the advertising delivery processing method in the virtual world in this specification are based on the same inventive concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0098] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] In the 1930s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many improvements to method flows today can be regarded as direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0100] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0101] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0102] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0103] It should be understood by those skilled in the art that one or more embodiments of this specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0108] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0111] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0112] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0113] The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims

1. A method for processing advertisement delivery in a virtual world, comprising: Inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension; Performing advertisement placement detection in a virtual world based on the advertisement placement probability determined by the multimodal emotion feature and the emotion level of each emotion dimension, wherein the emotion dimension includes an attention dimension, a pleasure dimension and / or a tension dimension, wherein the virtual world refers to a virtual simulation world based on decentralized collaboration and having an open economic system, wherein decentralized transactions are performed in the virtual world by generating non-homogeneous identifiers, and ownership of virtual assets is acquired through transactions; If the detection is passed, the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world is calculated, the candidate advertisement includes the asset advertisement of the virtual asset corresponding to the non-homogeneous identifier, and the virtual environment data includes the data of the virtual environment or virtual scene in the virtual world; Based on the matching degree, a delivery advertisement is determined from the candidate advertisements and delivery processing is performed.

2. According to the method for processing advertisement delivery in a virtual world according to claim 1, the emotion detection model comprises: The first network, the second network and the converged network; Accordingly, the emotion detection process includes: Inputting the EEG signal into the first network to perform feature extraction to obtain EEG signal features, and inputting the eye image into the second network to perform feature extraction to obtain eye image features; The EEG signal features and the eye image features are input into the fusion network for feature processing to obtain the multimodal emotion features and the emotion level.

3. According to the method for processing advertisement delivery in a virtual world as claimed in claim 2, the feature processing performed by the fusion network comprises: Performing feature fusion on the EEG signal feature and the eye image feature, and using the obtained fusion feature as the multimodal emotion feature; Emotion detection of the at least one emotion dimension is performed based on the EEG signal features and the eye image features to obtain an emotion level of the at least one emotion dimension.

4. The method for processing advertisement delivery in a virtual world according to claim 1, wherein the advertisement delivery detection in the virtual world is performed based on the advertisement delivery probability determined by the multimodal emotion feature and the emotion level of each emotion dimension, comprising: Detecting whether the emotion level of each emotion dimension is greater than or equal to the emotion level threshold of the corresponding emotion dimension, and whether the advertisement delivery probability determined based on the multimodal emotion feature is greater than a preset probability threshold; If yes, the test result is determined to be passed; If not, the test result is determined to be a failure.

5. According to the method for processing advertisement delivery in a virtual world as claimed in claim 4, the advertisement delivery probability is obtained by inputting the multimodal emotion feature into a delivery detection model to perform advertisement delivery adaptation calculation; in, The advertisement delivery adaptation calculation includes calculating the degree of adaptation for advertisement delivery at the current time point based on the multimodal emotion feature as the advertisement delivery probability.

6. The method for processing advertisement delivery in a virtual world according to claim 1, wherein the step of calculating the degree of matching between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world comprises: Extracting features of the key image included in the advertisement data and the environment image included in the virtual environment data to obtain advertisement key features and virtual environment features; The feature similarity between the advertisement key feature and the virtual environment feature is calculated as the matching degree.

7. The method for processing advertisement delivery in a virtual world according to claim 6, wherein the step of extracting features of the key image contained in the advertisement data and the environment image contained in the virtual environment data to obtain advertisement key features and virtual environment features comprises: The key image is input into a feature extraction model for feature extraction to obtain the key features of the advertisement, and the environment image is input into the feature extraction model for feature extraction to obtain the virtual environment features.

8. The method for processing advertisement delivery in a virtual world according to claim 7, wherein the feature extraction model is obtained by training in the following manner: Inputting the positive sample pairs and / or negative sample pairs in the training sample set into the residual network to be trained for feature extraction to perform feature extraction, and calculating the feature similarity of the output sample features; The training loss is calculated according to the feature similarity, and the parameters of the residual network are adjusted based on the training loss, and the feature extraction model is obtained after the training is completed.

9. The method for processing advertisement delivery in a virtual world according to claim 8, wherein the positive sample pair comprises a virtual environment image pair obtained by sampling from an image set of the same virtual environment; The negative sample pairs include virtual environment image pairs obtained by sampling from image sets of different virtual environments.

10. The method for processing advertisement delivery in a virtual world according to claim 9, wherein the image set of the virtual environment is generated in the following manner: The physical environment images obtained by multi-view image acquisition of the physical environment are input into a pre-trained neural radiation field model for three-dimensional image reconstruction, and the images corresponding to the three-dimensional virtual environment in the virtual world are output as the image set of the virtual environment.

11. The method for processing advertisement delivery in a virtual world according to claim 1, wherein determining an advertisement to be delivered from the candidate advertisements based on the matching degree and performing delivery processing comprises: Sorting the candidate advertisements in descending order of the matching degree, and selecting the candidate advertisements ranked in the top N positions; A candidate advertisement is randomly selected from the first N candidate advertisements as the delivered advertisement, and the delivered advertisement is rendered and output.

12. The method for processing advertisement delivery in a virtual world according to claim 1, wherein the EEG signal is collected by a signal sensor integrated in an access device of the virtual world; The eye image is collected by an image sensor integrated in the access device.

13. A virtual world advertising processing device, comprising: An emotion detection processing module is configured to input the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension; an advertisement delivery detection module, configured to perform advertisement delivery detection in a virtual world based on the advertisement delivery probability determined by the multimodal emotion feature and the emotion level of each emotion dimension, wherein the emotion dimension includes an attention dimension, a pleasure dimension and / or a tension dimension, wherein the virtual world refers to a virtual simulation world based on decentralized collaboration and having an open economic system, wherein decentralized transactions are performed in the virtual world by generating non-homogeneous identifiers, and ownership of virtual assets is obtained through transactions; a matching degree calculation module, configured to calculate the matching degree of the advertisement data of the candidate advertisement and the virtual environment data of the virtual world if the detection passes, the candidate advertisement comprising the asset advertisement of the virtual asset corresponding to the non-homogeneous identifier, and the virtual environment data comprising the data of the virtual environment or virtual scene in the virtual world; The delivery processing module is configured to determine a delivery advertisement from the candidate advertisements based on the matching degree and perform delivery processing.

14. A virtual world advertising processing device, comprising: processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: Inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension; Performing advertisement placement detection in a virtual world based on the advertisement placement probability determined by the multimodal emotion feature and the emotion level of each emotion dimension, wherein the emotion dimension includes an attention dimension, a pleasure dimension and / or a tension dimension, wherein the virtual world refers to a virtual simulation world based on decentralized collaboration and having an open economic system, wherein decentralized transactions are performed in the virtual world by generating non-homogeneous identifiers, and ownership of virtual assets is acquired through transactions; If the detection is passed, the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world is calculated, the candidate advertisement includes the asset advertisement of the virtual asset corresponding to the non-homogeneous identifier, and the virtual environment data includes the data of the virtual environment or virtual scene in the virtual world; Based on the matching degree, a delivery advertisement is determined from the candidate advertisements and delivery processing is performed.

15. A storage medium for storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the following process: Inputting the user's EEG signal and eye image into the emotion detection model for emotion detection processing to obtain multimodal emotion features and an emotion level of at least one emotion dimension; Performing advertisement placement detection in a virtual world based on the advertisement placement probability determined by the multimodal emotion feature and the emotion level of each emotion dimension, wherein the emotion dimension includes an attention dimension, a pleasure dimension and / or a tension dimension, wherein the virtual world refers to a virtual simulation world based on decentralized collaboration and having an open economic system, wherein decentralized transactions are performed in the virtual world by generating non-homogeneous identifiers, and ownership of virtual assets is acquired through transactions; If the detection is passed, the matching degree between the advertisement data of the candidate advertisement and the virtual environment data of the virtual world is calculated, the candidate advertisement includes the asset advertisement of the virtual asset corresponding to the non-homogeneous identifier, and the virtual environment data includes the data of the virtual environment or virtual scene in the virtual world; Based on the matching degree, a delivery advertisement is determined from the candidate advertisements and delivery processing is performed.