Virtual resource pushing method, device and equipment

Abstract

The embodiment of the invention discloses a virtual resource pushing method, device and equipment. The method comprises the following steps: acquiring an interactive image of a user; inputting a first frame image in the interactive images of the user into an interactive mode classification model to obtain an interactive mode of the user; obtaining an attention change function of the user based on the interaction mode of the user and the single-frame attention perception result of the user; based on the attention change function of the user, predicting peak time when the attention of the user reaches a peak value; based on the peak time, determining the pushing time of the virtual resources; and pushing the virtual resources at the pushing time.

Description

[0001] This application is a divisional application of the Chinese patent application. The application date of the original application is: March 10, 2022; the application number is: 202210231585.2; the name of the invention is: A method, device and equipment for pushing virtual resources. Technical Field

[0002] The present invention relates to the field of computer technology, and in particular to a method, device and equipment for pushing virtual resources. Background Art

[0003] With the development of network technology, there are more and more IoT (Internet of Things) devices with screens, including self-service cash registers, biometric payment devices, such as face-scanning payment devices, etc. Screens are generally used to display advertisements, virtual resources, and payment results. Virtual resources, such as coupons, are generally displayed after the user completes the payment. However, since the user has completed the payment, the user's attention has moved away from the screen, or even turned away, resulting in low exposure rate of virtual resources, low delivery efficiency, and poor display effect.

[0004] Based on this, a new method is needed to better display virtual resources and increase the exposure rate of virtual resources. Summary of the invention

[0005] The embodiments of this specification provide a method, device and equipment for pushing virtual resources, which are used to solve the following technical problems: the existing virtual resources have low exposure rate, low delivery efficiency and poor display effect.

[0006] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0007] The embodiment of this specification provides a method for pushing virtual resources, including:

[0008] Get the user's interactive image;

[0009] Inputting the first frame image in the user's interaction image into the interaction mode classification model to obtain the user's interaction mode;

[0010] Based on the interaction mode of the user and the single-frame attention perception result of the user, obtaining the attention change function of the user;

[0011] Predicting a peak time when the user's attention reaches a peak value based on the user's attention change function;

[0012] Based on the peak time, determining a push time of the virtual resource;

[0013] The virtual resource is pushed at the pushing time.

[0014] The embodiment of this specification also provides a virtual resource push device, including:

[0015] An acquisition module is used to acquire the user's interactive image;

[0016] An interaction mode judgment module inputs the first frame of the user's interaction image into an interaction mode classification model to obtain the user's interaction mode;

[0017] A process perception module, which obtains the user's attention change process based on the user's interaction mode and the user's single-frame attention perception result;

[0018] A prediction module, based on the user's attention change process, predicts the time when the user's attention reaches a peak value as the user's peak time;

[0019] The push module determines the time of pushing the virtual resources based on the peak time of the user, so as to push the virtual resources.

[0020] The embodiment of the present specification also provides an electronic device, including:

[0021] at least one processor; and,

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0024] Get the user's interactive image;

[0025] Inputting the first frame image in the user's interaction image into the interaction mode classification model to obtain the user's interaction mode;

[0026] Based on the interaction mode of the user and the single-frame attention perception result of the user, obtaining the attention change function of the user;

[0027] Predicting a peak time when the user's attention reaches a peak value based on the user's attention change function;

[0028] Based on the peak time, determining a push time of the virtual resource;

[0029] The virtual resource is pushed at the pushing time.

[0030] The embodiment of this specification provides a method for pushing virtual resources, which obtains the user's interactive image; inputs the first frame image in the user's interactive image into the interactive mode classification model to obtain the user's interactive mode; obtains the user's attention change function based on the user's interactive mode and the user's single-frame attention perception result; predicts the peak time when the user's attention reaches the peak based on the user's attention change function; determines the push time of virtual resources based on the peak time; and pushes the virtual resources at the push time. It can realize the intelligent push of virtual resources and improve the exposure rate and delivery efficiency of virtual resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 A schematic diagram of a method for pushing virtual resources provided in an embodiment of this specification;

[0033] Figure 2 A structural diagram of a method for pushing virtual resources provided in an embodiment of this specification;

[0034] Figure 3 A schematic diagram of a virtual resource pushing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0036] One way to push virtual resources is to receive virtual resources by scanning a code. This method is the most common, that is, displaying a QR code on the screen, and the user uses a mobile phone to scan the code to receive it. However, this link of scanning the code to receive virtual resources is too long, and it cannot be combined with the user's behavior information to achieve a thousand faces for a thousand people, resulting in a relatively low efficiency in issuing virtual resources. The method of receiving virtual resources after payment is significantly more efficient than the method of scanning the code to receive virtual resources. After the user completes the payment, the user's behavior information can generally be perceived, and virtual resources with a thousand faces can be pushed. The user can click to receive the virtual resources, which is more natural and convenient. However, the existing method of receiving virtual resources after payment has the problems of low exposure rate and low delivery efficiency. Therefore, the embodiment of this specification provides a new push method, which pushes virtual resources by paying attention to the changes in the user's attention and choosing the right time.

[0037] Figure 1 The schematic diagram of a method for pushing virtual resources provided in an embodiment of the present specification. The execution subject of the method may be a terminal device. Specifically, the terminal device is provided with a shooting function and can realize a payment function. Figure 1 As shown, the push method may include the following steps:

[0038] Step S101: Acquire the user's interaction image.

[0039] When users pay for virtual resources, it is necessary to store multiple frames of interactive images of users interacting in front of the display screen from the preparation of payment to the completion of payment. The device as the execution subject can use these interactive images as images to be processed to perceive the change process of user attention, so as to select the appropriate time to push virtual resources based on the perception results.

[0040] The user's interactive image can be stored locally on the terminal, or can be transmitted to a server for storage, or can be stored on a blockchain. Where the user's interactive image is stored does not constitute a limitation on the present application.

[0041] Step S103: inputting the first frame image in the user's interaction image into the interaction mode classification model to obtain the user's interaction mode.

[0042] When users make payments, there are usually two payment modes: code scanning payment mode and biometric payment mode. The attention process of these two payment modes is different, so it is necessary to distinguish the two payment modes.

[0043] In the embodiment of the present specification, the interaction mode classification model is a model obtained by training a convolutional neural network using the interaction modes of different users as input.

[0044] In one embodiment of the present specification, the interactive mode classification model is a classification model obtained by training based on the ResNet18 network structure and using binary cross entropy as the loss function. The input of the interactive mode classification model is the user's interactive image, and the output is the interactive mode type, which can specifically include a biometric payment mode and a code scanning payment mode.

[0045] The ResNet18 network structure is a convolutional neural network with a weight of 18, 17 convolutional layers and 1 fully connected layer. Cross entropy is generally used to find the gap between the target and the predicted value. The binary cross entropy is the predicted probability of belonging to two categories respectively. The loss function or cost function is a function that maps the value of a random event or its related random variables to a non-negative real number to represent the "risk" or "loss" of the random event. In applications, the loss function is usually associated with the optimization problem as a learning criterion, that is, the model is solved and evaluated by minimizing the loss function.

[0046] In one embodiment of the present specification, the training of the interaction mode classification model is to annotate the first frame of the user's historical interaction image, and the annotation is a biometric recognition mode or a QR code payment mode; the annotated interaction image is input into the ResNet18 network structure, and the binary cross entropy is used as the loss function. The loss function is iteratively optimized using the stochastic gradient descent method until convergence to obtain the interaction model classification model.

[0047] In the embodiments of this specification, stochastic gradient descent (SGD) is to calculate the gradient of the loss function and iteratively optimize the gradient of the loss function.

[0048] In the embodiment of the present specification, after the camera is turned on, the first frame in the user's interaction image is input into the trained interaction mode classification model, and the output is the user's interaction mode.

[0049] Step S105: Based on the user's interaction mode and the user's single-frame attention perception result, obtain the user's attention change function.

[0050] As mentioned above, the user's interactive image is composed of multiple frames, and the user's attention changes during the user's payment process. Therefore, based on the user's attention perception results of each frame, the user's attention change process can be known.

[0051] In the embodiment of the present specification, the single-frame attention perception result of the user is a weighted sum of the attention perception result, the head posture perception result and the body posture perception result of each frame of the user's image to obtain the attention score of each frame of the user's image;

[0052] in,

[0053] The attention perception result is to input each frame of the user's interactive image into the attention perception model to obtain the probability of the user looking at the screen;

[0054] The head posture perception result is the probability of the user's face facing the screen obtained by inputting each frame of the user's interactive image into the head posture perception model;

[0055] The body posture perception result is obtained by inputting each frame of the user's interactive image into a body posture perception model to obtain the probability that the user's body is facing the screen.

[0056] It should be noted that the interactive image refers to an image containing the user's image captured during the user's operation of the terminal. The attention perception model can calculate the probability that the user is looking at the screen for each frame of the image. The head posture perception model can calculate the probability that the user's face is facing the screen for each frame of the image. The body posture perception model can calculate the probability that the user's body is facing the screen for each frame of the image.

[0057] In the embodiment of this specification, the attention perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with eye regions marked as input;

[0058] The head posture perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with marked head areas as input;

[0059] The body posture perception model is a model obtained by training a convolutional neural network using interaction images of multiple users with upper body areas marked as input.

[0060] In the embodiment of this specification, the input of the attention perception model is an interactive image containing the user's eye area, and the output is the probability P1 of the user looking at the screen. The higher the probability, the greater the possibility that the user is looking at the screen and the higher the attention.

[0061] In one embodiment of the present specification, the attention perception model is an attention perception model obtained by training based on the ResNet18 network structure with binary cross entropy as the loss function.

[0062] In one embodiment of the present specification, the training of the attention perception model is to annotate the user's historical interaction images, where the annotation is the user's eye area; the annotated interaction image is input into the ResNet18 network structure, and the binary cross entropy is used as the loss function. The loss function is iteratively optimized using the stochastic gradient descent method until convergence to obtain the attention perception model.

[0063] In the embodiment of this specification, the input of the head posture perception model is an interactive image containing the user's head area, and the output is the probability P2 that the user's face is facing the screen. The higher the probability, the greater the possibility that the user is facing the screen and the higher the attention.

[0064] In one embodiment of the present specification, the head posture perception model is a head posture perception model obtained by training based on the ResNet18 network structure with binary cross entropy as the loss function.

[0065] In one embodiment of the present specification, the training of the head posture perception model is to annotate the user's historical interaction images, where the annotation is the user's head area; the annotated interaction image is input into the ResNet18 network structure, and the binary cross entropy is used as the loss function. The loss function is iteratively optimized using the stochastic gradient descent method until convergence to obtain the head posture perception model.

[0066] In the embodiment of this specification, the input of the body posture perception model is an interactive image containing the upper body of the user, and the output is the probability P3 that the user's body is facing the screen. The higher the probability, the greater the possibility that the user's body is facing the screen and the higher the attention.

[0067] In one embodiment of the present specification, the body posture perception model is a body posture perception model obtained by training based on the ResNet18 network structure with binary cross entropy as the loss function.

[0068] In one embodiment of the present specification, the training of the body posture perception model is to annotate the user's historical interaction images, where the annotations are the user's body areas; the annotated interaction images are input into the ResNet18 network structure, and the binary cross entropy is used as the loss function. The loss function is iteratively optimized using the stochastic gradient descent method until convergence to obtain the body posture perception model.

[0069] In the embodiment of the present specification, the user's attention perception results, body posture perception results and body posture perception results are weighted and summed, which is to fuse the results of multiple models. That is, the user's final attention score is calculated by the results of the three models in the embodiment of the present specification. In one embodiment of the present specification, the calculation formula of the attention score is P=a*P1+b*P2+c*P3, where a, b, c are weights, and a+b+c=1. In a specific embodiment, a, b, c are determined according to the degree of importance. Generally speaking, attention perception and head posture perception are considered to be more important. Therefore, a and b have larger weights, and c has the smallest weight.

[0070] In the embodiment of this specification, obtaining the user's attention change function based on the user's interaction mode and the user's single-frame attention perception result specifically includes:

[0071] Fitting the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution;

[0072] According to the prior mean and the prior variance of the interaction mode, based on the interaction mode of the user, correcting the mean and the variance to obtain a final mean and a final variance;

[0073] Based on the Gaussian distribution determined by the final mean and the final variance, the user's attention change function is determined.

[0074] In an embodiment of the present specification, multiple single-frame attention perception results of the user are fitted, wherein the single-frame attention perception results include the attention perception results of the attention rising process and the attention falling process, so as to obtain the mean of the Gaussian distribution and the variance of the Gaussian distribution by fitting. In one embodiment of the present specification, N attention scores are used for fitting to obtain the mean m and variance v, wherein N is the number of the aforementioned multiple-frame images, and in a specific embodiment, N is generally selected to be 10-15.

[0075] As mentioned above, the attention process of different interaction modes is different, and the corresponding prior means and prior variances are different. The prior means and prior variances are obtained by regressing the data of different interaction modes based on the existing data. The prior mean is denoted as m' and the prior variance is denoted as v'.

[0076] In the embodiment of the present specification, the interaction mode includes a code scanning payment mode and a biometric payment mode, and the prior mean and prior variance include the prior mean of the code scanning payment mode and the prior variance of the code scanning payment mode, the prior mean of the biometric payment mode and the prior variance of the biometric payment mode.

[0077] In the embodiment of the present specification, the prior mean and the prior variance of the interaction mode are corrected based on the interaction mode of the user to obtain the final mean and the final variance, specifically including:

[0078] Determining a priori mean and a priori variance of the user based on the interaction pattern of the user;

[0079] Performing a weighted summation on the prior mean and the mean of the Gaussian distribution to obtain a final mean;

[0080] The prior variance and the variance of the Gaussian distribution are weightedly summed to obtain the final variance.

[0081] The mean m and variance v obtained by fitting are corrected by the prior mean m' and the prior variance v' to obtain the final mean and the final variance. In one embodiment of the present specification, the final mean = 0.9*m + 0.1*m', and the final variance = 0.9*v + 0.1*v'.

[0082] Step S107: Based on the user's attention change function, predict the peak time when the user's attention reaches a peak.

[0083] Based on the final mean and final variance obtained in the embodiments of this specification, a Gaussian distribution for attention process estimation can be obtained, and thus the peak time can be determined based on the Gaussian distribution.

[0084] Step S109: Determine the push time of the virtual resource based on the peak time.

[0085] According to the peak time obtained above, it can be determined whether the user is currently in a period of increasing attention, or a period of decreasing attention, or has reached a peak.

[0086] In the embodiment of this specification, determining the push time of the virtual resource based on the peak time specifically includes:

[0087] Determine, based on the peak time of the user, a function of the user's attention trend changing over time;

[0088] Determine the user's attention at the current moment in real time according to the function;

[0089] If the user attention reaches a peak or starts to decrease, the current time is used as the push time of the virtual resource.

[0090] Of course, if the user's attention is on the rise, the user's attention is continuously predicted until the attention reaches a peak, and then the virtual resources are pushed.

[0091] Step S111: Pushing the virtual resource at the pushing time.

[0092] In the embodiment of this specification, the method further comprises:

[0093] Record whether the user clicks on the virtual resource.

[0094] Regarding the method for pushing virtual resources provided in the embodiments of this specification, it should be particularly noted that for different interaction modes, namely the code scanning payment mode and the biometric payment mode, the interaction mode judgment model, the attention perception model, the head posture perception model and the body posture perception model are all the same. The only difference lies in the specific data of the prior mean and the prior variance.

[0095] In order to further understand the method for pushing virtual resources provided in the embodiments of this specification, Figure 2 A structural diagram of a method for pushing virtual resources is provided. Figure 2 A structural diagram of a method for pushing virtual resources provided in an embodiment of this specification, such as Figure 2 As shown, based on the user's interaction image, the user interaction mode is judged to obtain the user's interaction mode; the user's single-frame attention perception result is further determined; based on the user's single-frame attention perception result, the user's attention change function is obtained; finally, based on the user's attention change function, the push time of the virtual resource is determined to push the virtual resource.

[0096] The method for pushing virtual resources provided in the embodiments of this specification can realize intelligent pushing of virtual resources and improve the exposure rate and delivery efficiency of virtual resources.

[0097] The above content describes in detail a method for pushing virtual resources. Correspondingly, this specification also provides a device for pushing virtual resources, such as Figure 3 shown. Figure 3 A schematic diagram of a virtual resource push device provided in an embodiment of this specification, the device comprising:

[0098] An acquisition module 301 acquires a user's interaction image;

[0099] The interaction mode determination module 303 inputs the first frame image in the user's interaction image into the interaction mode classification model to obtain the user's interaction mode;

[0100] The process perception module 305 obtains the user's attention change function based on the user's interaction mode and the user's single-frame attention perception result;

[0101] Prediction module 307, predicting the peak time when the user's attention reaches a peak value based on the user's attention change function;

[0102] A determination module 309 determines a push time of virtual resources based on the peak time;

[0103] The push module 311 pushes the virtual resource at the push time.

[0104] In the embodiment of the present specification, the interaction mode classification model is a model obtained by training a convolutional neural network using the interaction modes of different users as input.

[0105] In the embodiment of the present specification, the interaction mode classification model is a model obtained by training a convolutional neural network using the interaction modes of different users as input.

[0106] In the embodiment of the present specification, the single-frame attention perception result of the user is a weighted sum of the attention perception result, the head posture perception result and the body posture perception result of each frame of the user's image to obtain the attention score of each frame of the user's image;

[0107] in,

[0108] The attention perception result is to input each frame of the user's interactive image into the attention perception model to obtain the probability of the user looking at the screen;

[0109] The head posture perception result is the probability of the user's face facing the screen obtained by inputting each frame of the user's interactive image into the head posture perception model;

[0110] The body posture perception result is obtained by inputting each frame of the user's interactive image into a body posture perception model to obtain the probability that the user's body is facing the screen.

[0111] In the embodiment of this specification, the attention perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with eye regions marked as input;

[0112] The head posture perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with marked head areas as input;

[0113] The body posture perception model is a model obtained by training a convolutional neural network using interaction images of multiple users with upper body areas marked as input.

[0114] In the embodiment of this specification, obtaining the user's attention change function based on the user's interaction mode and the user's single-frame attention perception result specifically includes:

[0115] Fitting the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution;

[0116] According to the prior mean and the prior variance of the interaction mode, based on the interaction mode of the user, correcting the mean and the variance to obtain a final mean and a final variance;

[0117] Based on the Gaussian distribution determined by the final mean and the final variance, the user's attention change function is determined.

[0118] In the embodiment of the present specification, the interaction mode includes a code scanning payment mode and a biometric payment mode, and the prior mean and prior variance include the prior mean of the code scanning payment mode and the prior variance of the code scanning payment mode, the prior mean of the biometric payment mode and the prior variance of the biometric payment mode.

[0119] In the embodiment of the present specification, the prior mean and the prior variance of the interaction mode are corrected based on the interaction mode of the user to obtain the final mean and the final variance, specifically including:

[0120] Determining a priori mean and a priori variance of the user based on the interaction pattern of the user;

[0121] Performing a weighted summation on the prior mean and the mean of the Gaussian distribution to obtain a final mean;

[0122] The prior variance and the variance of the Gaussian distribution are weightedly summed to obtain the final variance.

[0123] In the embodiment of this specification, determining the push time of the virtual resource based on the peak time specifically includes:

[0124] Determine, based on the peak time of the user, a function of the user's attention trend changing over time;

[0125] Determine the user's attention at the current moment in real time according to the function;

[0126] If the user attention reaches a peak or starts to decrease, the current time is used as the push time of the virtual resource.

[0127] In the embodiment of this specification, the device further comprises:

[0128] The recording module 313 records whether the user clicks on the virtual resource.

[0129] The embodiment of the present specification also provides an electronic device, including:

[0130] at least one processor; and,

[0131] a memory communicatively connected to the at least one processor; wherein,

[0132] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0133] Get the user's interactive image;

[0134] Inputting the first frame image in the user's interaction image into the interaction mode classification model to obtain the user's interaction mode;

[0135] Based on the interaction mode of the user and the single-frame attention perception result of the user, obtaining the attention change function of the user;

[0136] Predicting a peak time when the user's attention reaches a peak value based on the user's attention change function;

[0137] Based on the peak time, determining a push time of the virtual resource;

[0138] The virtual resource is pushed at the pushing time.

[0139] In the embodiment of the present specification, the interaction mode classification model is a model obtained by training a convolutional neural network using the interaction modes of different users as input.

[0140] In the embodiment of the present specification, the single-frame attention perception result of the user is a weighted sum of the attention perception result, the head posture perception result and the body posture perception result of each frame of the user's image to obtain the attention score of each frame of the user's image;

[0141] in,

[0142] The attention perception result is to input each frame of the user's interactive image into the attention perception model to obtain the probability of the user looking at the screen;

[0143] The head posture perception result is the probability of the user's face facing the screen obtained by inputting each frame of the user's interactive image into the head posture perception model;

[0144] The body posture perception result is obtained by inputting each frame of the user's interactive image into a body posture perception model to obtain the probability that the user's body is facing the screen.

[0145] In the embodiment of this specification, the attention perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with eye regions marked as input;

[0146] The head posture perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with marked head areas as input;

[0147] The body posture perception model is a model obtained by training a convolutional neural network using interaction images of multiple users with upper body areas marked as input.

[0148] In the embodiment of this specification, obtaining the user's attention change function based on the user's interaction mode and the user's single-frame attention perception result specifically includes:

[0149] Fitting the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution;

[0150] According to the prior mean and the prior variance of the interaction mode, based on the interaction mode of the user, correcting the mean and the variance to obtain a final mean and a final variance;

[0151] Based on the Gaussian distribution determined by the final mean and the final variance, the user's attention change function is determined.

[0152] In the embodiment of the present specification, the interaction mode includes a code scanning payment mode and a biometric payment mode, and the prior mean and prior variance include the prior mean of the code scanning payment mode and the prior variance of the code scanning payment mode, the prior mean of the biometric payment mode and the prior variance of the biometric payment mode.

[0153] In the embodiment of the present specification, the prior mean and the prior variance of the interaction mode are corrected based on the interaction mode of the user to obtain the final mean and the final variance, specifically including:

[0154] Determining a priori mean and a priori variance of the user based on the interaction pattern of the user;

[0155] Performing a weighted summation on the prior mean and the mean of the Gaussian distribution to obtain a final mean;

[0156] The prior variance and the variance of the Gaussian distribution are weightedly summed to obtain the final variance.

[0157] In the embodiment of this specification, determining the push time of the virtual resource based on the peak time specifically includes:

[0158] Determine, based on the peak time of the user, a function of the user's attention trend changing over time;

[0159] Determine the user's attention at the current moment in real time according to the function;

[0160] If the user attention reaches a peak or starts to decrease, the current time is used as the push time of the virtual resource.

[0161] In the embodiment of this specification, the method further comprises:

[0162] Record whether the user clicks on the virtual resource.

[0163] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0164] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0165] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0166] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0167] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0168] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0169] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0170] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0171] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0176] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0178] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0179] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0180] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for pushing virtual resources, the method comprising: Get the user's interactive image; Based on the interactive image, determining a single-frame attention perception result of the user for each frame of the interactive image; Based on the single-frame attention perception result, obtaining an attention change function of the user; Determining a push time of a virtual resource based on the user's attention change function; The virtual resource is pushed at the pushing time.

2. The method according to claim 1, wherein determining the user's single-frame attention perception result for each frame of the interactive image comprises: Determine the user's attention score for each frame of the image based on at least a portion of the user's attention perception result, the head posture perception result, and the body posture perception result for each frame of the image; in, The attention perception result includes inputting each frame of the user's interactive image into the attention perception model to obtain the probability of the user looking at the screen; The head posture perception result includes inputting each frame of the user's interactive image into a head posture perception model to obtain a probability that the user's face is facing the screen; The body posture perception result includes inputting each frame of the user's interactive image into a body posture perception model to obtain the probability that the user's body is facing the screen.

3. The method according to claim 2, wherein determining the user's attention score for each frame of the image based on at least a part of the user's attention perception result, head posture perception result, and body posture perception result for each frame of the image comprises: The attention perception result, head posture perception result and body posture perception result of each frame of the user are weightedly summed to determine the user's attention score for each frame of the image.

4. The method according to claim 2, wherein the attention perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with eye regions labeled as input; The head posture perception model is a model obtained by training a convolutional neural network using the interactive images of multiple users with marked head areas as input; The body posture perception model is a model obtained by training a convolutional neural network using interaction images of multiple users with upper body areas marked as input.

5. The method according to claim 1, before obtaining the user's attention change function based on the single-frame attention perception result, further comprising: Inputting the first frame image in the interaction image into an interaction mode classification model to obtain the interaction mode of the user; The obtaining the user's attention change function based on the single-frame attention perception result specifically includes: Based on the user's interaction mode and the user's single-frame attention perception result, an attention change function of the user is obtained.

6. In the method as claimed in claim 5, the interaction mode classification model is a model obtained by training a convolutional neural network using the interaction modes of different users as input.

7. The method according to claim 5, wherein obtaining the user's attention change function based on the user's interaction mode and the user's single-frame attention perception result specifically comprises: Fitting the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution; According to the prior mean and the prior variance of the interaction mode, the mean and the variance are corrected to obtain a final mean and a final variance; Based on the Gaussian distribution determined by the final mean and the final variance, the user's attention change function is determined.

8. According to the method as claimed in claim 7, the interaction mode includes a code scanning payment mode, the prior mean includes the prior mean of the code scanning payment mode, and the prior variance includes the prior variance of the code scanning payment mode; or, the interaction mode includes a biometric payment mode, the prior mean includes the prior mean of the biometric payment mode, and the prior variance includes the prior variance of the biometric payment mode.

9. The method according to claim 7, wherein the single-frame attention perception result comprises an attention perception result of an attention rising process and an attention perception result of an attention falling process; The fitting of the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution specifically includes: Based on the attention perception result of the attention rising process and the attention perception result of the attention falling process, the mean of the Gaussian distribution and the variance of the Gaussian distribution are fitted.

10. The method according to claim 7, wherein the step of correcting the prior mean and the prior variance of the interaction mode to obtain a final mean and a final variance comprises: Determining a priori mean and a priori variance of the user based on the interaction pattern of the user; Performing a weighted summation on the prior mean and the mean of the Gaussian distribution to obtain a final mean; The prior variance and the variance of the Gaussian distribution are weightedly summed to obtain the final variance.

11. The method according to claim 1, wherein determining the push time of the virtual resource based on the user's attention change function specifically comprises: Predicting a peak time when the user's attention reaches a peak based on the user's attention change function; Based on the peak time, a push time of the virtual resource is determined.

12. The method according to claim 11, before obtaining the user's attention change function based on the single-frame attention perception result, further comprising: Inputting the first frame image in the interaction image into an interaction mode classification model to obtain the interaction mode of the user; The obtaining the user's attention change function based on the single-frame attention perception result specifically includes: Fitting the single-frame attention perception result of the user to obtain the mean and variance of the Gaussian distribution; According to the prior mean and the prior variance of the interaction mode, the mean and the variance are corrected to obtain a final mean and a final variance; Determining an attention change function of the user based on a Gaussian distribution determined by the final mean and the final variance; The predicting, based on the user's attention change function, the peak time when the user's attention reaches a peak value specifically includes: Based on the final mean and the final variance, obtaining a target Gaussian distribution for estimating an attention process; The peak time is predicted based on the target Gaussian distribution.

13. The method according to claim 11, wherein determining the push time of the virtual resource based on the peak time specifically comprises: Determine, based on the peak time of the user, a function of the user's attention trend changing over time; Determine the user's attention at the current moment in real time according to the function; If the user attention reaches a peak or starts to decrease, the current time is used as the push time of the virtual resource.

14. The method according to claim 13, after determining the user's attention degree at the current moment in real time according to the function, further comprising: If the user's attention is in an increasing period, the time when the attention reaches a peak value is determined as the push time of the virtual resource.

15. The method of claim 1, further comprising: Record whether the user clicks on the virtual resource.

16. A virtual resource push device, the device comprising: An acquisition module is used to acquire the user's interactive image; A process perception module, based on the interactive image, determines a single-frame attention perception result of the user for each frame of the interactive image; Based on the single-frame attention perception result, obtaining the attention change function of the user; A determination module, which determines the push time of the virtual resource based on the user's attention change function; A push module pushes the virtual resource at the push time.

17. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Get the user's interactive image; Based on the interactive image, determining a single-frame attention perception result of the user for each frame of the interactive image; Based on the single-frame attention perception result, obtaining an attention change function of the user; Determining a push time of a virtual resource based on the user's attention change function; The virtual resource is pushed at the pushing time.

Images