Virtual prop interaction method, device and computer equipment

Abstract

The application relates to the field of network media, in particular to a virtual prop interaction method and device and computer equipment. The method comprises the following steps: a multimedia playing interface is displayed, and a target object is displayed in the multimedia playing interface; an interaction entrance is displayed in the multimedia playing interface; in response to a triggering operation on the interaction entrance, an interaction animation is displayed in a region range corresponding to the target object in the multimedia playing interface, and the interaction animation comprises a virtual prop which interacts in a manner matched with a facial expression of the target object. The method can improve the diversity of interaction manners.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a virtual prop interaction method, device and computer equipment. Background Art

[0002] With the development of Internet technology, more and more users are using multimedia applications to play multimedia content. For example, users can play TV series, movies, and variety shows through multimedia applications.

[0003] Currently, when watching multimedia content, users can comment on a target object in the multimedia content by sending barrage comments. For example, a user can comment on the male protagonist in a TV series through barrage comments. However, this method only provides a limited number of ways for users to interact with the target object. Therefore, how to effectively increase the diversity of ways for users to interact with the target object in multimedia content has become an urgent problem to be solved in the existing technology. Summary of the Invention

[0004] Based on this, it is necessary to provide a virtual prop interaction method, device, computer equipment and storage medium that can enhance the diversity of interaction methods to address the above technical problems.

[0005] A virtual prop interaction method, the method comprising:

[0006] Displaying a multimedia playback interface, wherein the multimedia playback interface displays a target object;

[0007] Displaying an interactive entrance in the multimedia playback interface;

[0008] In response to a triggering operation on the interactive entrance, an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface, wherein the interactive animation includes a virtual prop that interacts in a manner that matches the facial expression of the target object.

[0009] A virtual prop interactive device, comprising:

[0010] An interface display module, configured to display a multimedia playback interface, wherein the multimedia playback interface displays a target object;

[0011] An entry display module, configured to display an interactive entry in the multimedia playback interface;

[0012] An interactive module is configured to display an interactive animation within an area corresponding to the target object in the multimedia playback interface in response to a triggering operation on the interactive portal, wherein the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression of the target object.

[0013] In one embodiment, the interactive module is also used to detect the area where the target part of the target object played in the multimedia playback interface is located, and the facial expression of the target object; obtain the target interaction method that matches the facial expression of the target object; and display an interactive animation including virtual props that interact according to the target interaction method within the area corresponding to the area where the target part is located.

[0014] In one embodiment, the target part includes a facial part, and the interactive module is further used to obtain facial features of the target object; wherein the facial features are extracted from a facial image collected for the target object; the facial key points of the target object are determined based on the facial features; based on the facial key points, the area where the facial part of the target object is located is determined, and the facial expression of the target object is determined.

[0015] In one embodiment, the target part includes a facial part, and the interactive module is further used to detect the playback progress of the multimedia content played by the multimedia playback interface; filter out target recognition results from a preset recognition result library according to the playback progress, and determine the area where the facial part of the target object is located at the current moment according to the target recognition results, and determine the facial expression of the target object.

[0016] In one embodiment, the virtual prop interaction device is also used to obtain multimedia content played by a multimedia application, and perform frame extraction on the multimedia content according to a preset frame extraction frequency to obtain multiple extracted target video frames; for each target video frame, the target object in the current target video frame is identified to obtain a recognition result; wherein the recognition result includes the area where the facial part of the target object is located, and the facial expression of the target object; the recognition results corresponding to each target video frame are stored to obtain a recognition result library.

[0017] In one embodiment, the virtual prop interaction device is further used to perform frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain extracted video frames; perform frame missing test on the extracted video frames according to the frame extraction frequency to obtain test results, and perform supplementary frame extraction processing on the multimedia content based on the test results to obtain supplementary video frames; based on the extracted video frames and the supplementary video frames, obtain multiple frames of target video frames.

[0018] In one embodiment, the virtual prop interaction device is also used to determine the playback progress corresponding to each target video frame, and divide the playback progress of the target video frame into multiple playback progress intervals according to a preset time window and time window movement step; corresponding to each playback progress interval, determine the difference between the recognition results of each target video frame falling within the current playback progress interval, and merge the recognition results whose differences are less than the difference threshold; store the merged recognition results to obtain a recognition result library.

[0019] In one embodiment, the virtual prop interaction device is further used to interpolate the recognition results in the recognition result library according to the frame extraction frequency and the frame rate of the multimedia content played by the multimedia playback interface to obtain an interpolation result; determine the correspondence between the position information of the target object in the multimedia content and the playback progress according to the interpolation result; determine the trajectory curve of the virtual prop according to the correspondence, and smooth and optimize the trajectory curve to obtain an optimized trajectory curve; display an interactive animation within the area corresponding to the target object in the multimedia playback interface, wherein the virtual prop in the interactive animation moves according to the optimized trajectory curve.

[0020] In one embodiment, the interaction mode includes the display effect, swing amplitude and swing frequency of the virtual prop. The interaction module is further used to determine that when the facial expression of the target object belongs to the first expression category, the target interaction mode that matches the facial expression of the target object is the first interaction mode; when the facial expression of the target object belongs to the second expression category, the target interaction mode that matches the facial expression of the target object is determined to be the second interaction mode; wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual prop in the first interaction mode is different from that of the virtual prop in the second interaction mode.

[0021] In one embodiment, the interaction module is further configured to respond to a triggering operation on the interaction entrance by displaying an interactive animation in which a virtual prop appears from the interaction entrance, moves toward the facial area of ​​the target object, and interacts.

[0022] In one embodiment, the interactive module is further used to display an interactive animation. There are multiple virtual props in the interactive animation. After the multiple virtual props move to the area where the facial parts of the target object are located, they interact around the facial parts of the target object, and each virtual prop interacts in an interactive manner that matches the facial expression of the target object.

[0023] In one embodiment, the interactive module is also used to enter the interactive space in response to a triggering operation on the interactive entrance, and display interactive controls of virtual props in the interactive space; in response to a specified operation on the interactive control, display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

[0024] In one embodiment, the interactive module is also used to determine all login accounts currently entering the interactive space, and determine the first operation times of all login accounts triggering a specified operation for the interactive control; wherein, the login account is an account for viewing the multimedia content played by the multimedia playback interface; an interactive animation is displayed within the area corresponding to the target object in the multimedia playback interface; wherein, the number of virtual props included in the interactive animation is updated in real time as the first operation times accumulate.

[0025] In one embodiment, the interactive module is further used to determine the number of virtual props displayed in the interactive animation and the display color of the virtual props based on the first operation number; and display the interactive animation within the area corresponding to the target object in the multimedia playback interface based on the display color of the virtual props and the number of virtual props.

[0026] In one embodiment, the virtual object interaction device is also used to determine the number of all login accounts that jointly trigger a specified operation for the interactive control; and display the number of accounts and the first number of operations at a first position corresponding to the interactive control.

[0027] In one embodiment, the virtual object interaction device is further used to determine a second operation number of the local login account triggering the specified operation for the interactive control; and display the second operation number at a second position corresponding to the interactive control.

[0028] In one embodiment, the virtual object interaction device is also used to determine the storage quantity of the virtual props included in the local login account; display the storage quantity at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the second operation number accumulates.

[0029] In one embodiment, the virtual object interaction device is also used to display the resource value required to replenish the storage quantity of the virtual prop when the storage quantity is updated to zero as the second operation number accumulates, and provide a resource value replenishment entry; the resource value replenishment entry is used to implement resource transfer operations.

[0030] In one embodiment, the interactive control includes a first interactive control and a second interactive control, and the interactive module is further used to enter the interactive space in response to a trigger operation on the interactive entrance, and display the first interactive control in the interactive space; in response to a first trigger operation on the first interactive control, cancel the display of the first interactive control, and switch to displaying the second interactive control, and display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

[0031] In one embodiment, the virtual object interaction device is also used to display an interactive countdown prompt animation at a fourth position corresponding to the second interactive control in response to a second trigger operation on the second interactive control; when the interactive countdown prompt animation ends, the second interactive control and the interactive animation are canceled.

[0032] In one embodiment, the entry display module is further configured to display an interactive entry in the multimedia playback interface when it is determined that the multimedia content in the multimedia playback interface has been played to a specified position.

[0033] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Displaying a multimedia playback interface, wherein the multimedia playback interface displays a target object;

[0035] Displaying an interactive entrance in the multimedia playback interface;

[0036] In response to a triggering operation on the interactive entrance, an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface, wherein the interactive animation includes a virtual prop that interacts in a manner that matches the facial expression of the target object.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0038] Displaying a multimedia playback interface, wherein the multimedia playback interface displays a target object;

[0039] Displaying an interactive entrance in the multimedia playback interface;

[0040] In response to a triggering operation on the interactive entrance, an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface, wherein the interactive animation includes a virtual prop that interacts in a manner that matches the facial expression of the target object.

[0041] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following steps: displaying a multimedia playback interface, wherein a target object is displayed in the multimedia playback interface; displaying an interactive portal in the multimedia playback interface; and displaying an interactive animation in an area corresponding to the target object in the multimedia playback interface in response to a triggering operation on the interactive portal, wherein the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression of the target object.

[0042] The above-mentioned virtual props interaction method, device, computer equipment, storage medium and computer program can display the target object and the interaction entrance through the multimedia playback interface by displaying the multimedia playback interface. By displaying the interaction entrance, in response to the triggering operation of the interaction entrance, an interactive animation of the virtual props that interacts in an interactive manner that matches the facial expression of the target object is displayed within the range corresponding to the target object. Since an interactive animation of the virtual props that interacts in an interactive manner that matches the facial expression of the target object can be displayed within the range corresponding to the target object in the multimedia playback interface, compared to the traditional way of participating in the interaction only by sending barrages, the present application can not only establish a strong connection with the target object, but also improve the user experience and increase the diversity of interaction methods with the target object in the multimedia content.

[0043] Furthermore, the present invention can display interactive portals while the user is watching multimedia content, allowing the user to conveniently interact with the target object based on the displayed interactive portal. This not only improves the efficiency of interaction but also enables the user to experience a good emotional resonance, greatly enhancing the effectiveness of the interaction. Furthermore, since the user only needs to trigger the displayed interactive portal to interact with the target object through virtual props, the efficiency of human-computer interaction is greatly improved.

[0044] A virtual prop interaction method, the method comprising:

[0045] In response to a triggering operation on an interactive entry in a multimedia playback interface, obtaining a facial expression of a target subject in the multimedia content to be played;

[0046] For the multimedia content to be played, drawing an interactive animation within a region corresponding to the target object, wherein the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression;

[0047] The multimedia content to be played and the corresponding interactive animation are superimposed.

[0048] A virtual prop interactive device, comprising:

[0049] An acquisition module, configured to acquire the facial expression of a target object in the multimedia content to be played in response to a triggering operation on an interactive entry in the multimedia playback interface;

[0050] a drawing module, configured to draw, for the multimedia content to be played, an interactive animation corresponding to the target object within a region corresponding to the target object, wherein the interactive animation includes a virtual prop that interacts in a manner that matches the facial expression;

[0051] The superposition module is used to superimpose the multimedia content to be played and the corresponding interactive animation.

[0052] In one embodiment, the acquisition module is further used to determine the playback progress corresponding to each frame of multimedia content to be played; based on the playback progress corresponding to each frame of multimedia content to be played, filter out target recognition results from a preset recognition result library; and based on the target recognition results, determine the facial expression of the target object in each frame of multimedia content to be played.

[0053] In one embodiment, the drawing module is further configured to determine candidate display areas for the virtual props in each frame of the multimedia content to be played based on a target portion of a target object in each frame of the multimedia content to be played; determine a trajectory curve for the virtual prop based on the candidate display areas, and perform smoothing optimization on the trajectory curve to obtain an optimized trajectory curve; determine a target display area for the virtual props in each frame of the multimedia content to be played based on the optimized trajectory curve, and draw the virtual props within the target display area so as to interact in an interactive manner that matches the facial expression to obtain an interactive animation.

[0054] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0055] In response to a triggering operation on an interactive entry in a multimedia playback interface, obtaining a facial expression of a target subject in the multimedia content to be played;

[0056] For the multimedia content to be played, drawing an interactive animation within a region corresponding to the target object, wherein the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression;

[0057] The multimedia content to be played and the corresponding interactive animation are superimposed.

[0058] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0059] In response to a triggering operation on an interactive entry in a multimedia playback interface, obtaining a facial expression of a target subject in the multimedia content to be played;

[0060] For the multimedia content to be played, drawing an interactive animation within a region corresponding to the target object, wherein the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression;

[0061] The multimedia content to be played and the corresponding interactive animation are superimposed.

[0062] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following steps: in response to a trigger operation on an interactive entry in a multimedia playback interface, obtaining a facial expression of a target object in multimedia content to be played; for the multimedia content to be played, drawing an interactive animation within a region corresponding to the target object, wherein the interactive animation includes a virtual prop that interacts in an interactive manner matching the facial expression; and superimposing the multimedia content to be played and the corresponding interactive animation.

[0063] The above-mentioned virtual prop interaction method, device, computer device, storage medium, and computer program can obtain multimedia content to be played by responding to a trigger operation on an interactive portal. By obtaining the multimedia content to be played, the facial expression of the target object in the multimedia content to be played can be obtained, and an interactive animation can be drawn based on the facial expression. By drawing the interactive animation, the multimedia content to be played and the corresponding interactive animation can be superimposed and then played. In this way, an interactive animation including a virtual prop that interacts in an interactive manner that matches the target object's facial expression can be displayed within the range corresponding to the target object in the multimedia playback interface. This not only establishes a strong connection with the target object, but also improves the user experience and increases the diversity of interaction methods with the target object in the multimedia content. Moreover, this device can display the interactive animation while not disturbing the user watching the multimedia content, enabling the user to obtain a good emotional resonance experience and greatly improving the effectiveness of the interaction. In addition, since the user only needs to trigger the displayed interactive portal to interact with the target object through the virtual prop, the efficiency of human-computer interaction is also greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A diagram of an application environment of a virtual prop interaction method in one embodiment;

[0065] Figure 2 1 is a flow chart of a virtual prop interaction method according to an embodiment;

[0066] Figure 3 is a schematic diagram of a target object in one embodiment;

[0067] Figure 4 is a schematic diagram of an interactive control in one embodiment;

[0068] Figure 5 A schematic diagram of facial key points according to an embodiment;

[0069] Figure 6 is a schematic diagram of facial expressions in one embodiment;

[0070] Figure 7 A schematic diagram of an interactive method of a virtual prop in one embodiment;

[0071] Figure 8 A schematic diagram of a flow chart for determining an interaction method in one embodiment;

[0072] Figure 9 A schematic diagram of the interaction principle of virtual props in one embodiment;

[0073] Figure 10 A schematic diagram of the interactive effect of virtual props in one embodiment;

[0074] Figure 11 A schematic diagram showing changes in the display color of a virtual item in one embodiment;

[0075] Figure 12 is a schematic diagram of a multimedia playback interface in one embodiment;

[0076] Figure 13 is a schematic diagram of an interactive control in one embodiment;

[0077] Figure 14 A schematic diagram of an interactive countdown prompt animation in one embodiment;

[0078] Figure 15 Schematic diagram of a flow chart of a virtual prop interaction method in a specific embodiment;

[0079] Figure 16 The process intention of virtual prop interaction in another embodiment;

[0080] Figure 17 The following is a flow diagram of a virtual prop interaction method according to another embodiment;

[0081] Figure 18 Schematic diagram of interaction of a virtual props interactive system in one embodiment;

[0082] Figure 19 A schematic diagram of a framework for virtual prop interaction in one embodiment;

[0083] Figure 20 A logic flow chart for implementing interactive animation in one embodiment;

[0084] Figure 21 A schematic diagram of determining an interaction method in one embodiment;

[0085] Figure 22 is a structural block diagram of a virtual prop interaction device in one embodiment;

[0086] Figure 23 is a structural block diagram of a virtual prop interaction device in another embodiment;

[0087] Figure 24 is a diagram of the internal structure of a computer device in one embodiment;

[0088] Figure 25 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0090] Figure 1 FIG. 1 is an application environment diagram of a virtual prop interaction method in an embodiment. Figure 1 The virtual prop interaction method is applied to a virtual prop interaction system 100. The virtual prop interaction system 100 includes a terminal 102 and a server 104. A multimedia application runs on the terminal 102, which can display a multimedia playback interface and a target object through the multimedia playback interface. The multimedia playback interface can display an interactive portal. When a user triggers the interactive portal to enter the interactive space, the server 104 can perform frame-by-frame recognition on the video stream played by the multimedia interface to obtain the target object's location information and facial expression. The server 104 then returns the location information and facial expression to the terminal 102, so that the terminal 102 displays an interactive animation including a virtual prop that interacts in an interactive manner that matches the target object's facial expression within the area corresponding to the target object in the multimedia playback interface based on the received location information and facial expression. The target object is an object displayed on the multimedia playback interface, and can specifically be a person, animal, cartoon character, etc. in the multimedia content played on the multimedia playback interface. The multimedia playback interface can display multiple target objects, each with its own shape and volume.

[0091] The terminal 102 may be provided with a multimedia application client, which may be a video client, a browser client, an information stream client, or the like. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 may be, but is not limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smartwatch, an in-vehicle terminal, a smart home appliance, or the like. The terminal 102 and the server 104 may be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions on the number of terminals 102 and servers 104.

[0092] This application relates to the field of artificial intelligence (AI). For example, this application can use artificial intelligence to recognize the facial expressions of target objects. Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0093] The present application can be applied in the field of network media. For example, a multimedia playback application is running in the terminal 102, and the terminal 102 can interact with a target object in the multimedia content through the multimedia playback application.

[0094] The present application can also be applied to the field of Internet of Vehicles. For example, when the terminal 102 is a vehicle-mounted terminal, multimedia content can be played through the vehicle-mounted terminal, thereby interacting with the target object in the multimedia content based on the vehicle-mounted terminal.

[0095] In one embodiment, Figure 2 As shown, a virtual prop interaction method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0096] Step S202: displaying a multimedia playback interface, wherein the target object is displayed in the multimedia playback interface.

[0097] Among them, the multimedia playback interface is the interface displayed when the multimedia application plays multimedia content. The multimedia playback interface can specifically be a live broadcast interface, a video playback interface, or an instant video playback interface, etc. The target object is an object in the multimedia content played by the multimedia playback interface, and the object can specifically be a person, animal, cartoon character, etc. in the multimedia content. Multiple target objects can be displayed in the multimedia playback interface, wherein each target object has its own shape and volume. When the multimedia playback interface can specifically be a live broadcast interface, the target object can specifically be an object in the live broadcast; when the multimedia playback interface is a video playback interface, the target object can specifically be an object in the recorded video; when the multimedia playback interface is an instant video playback interface, the target object can specifically be an object during a video call.

[0098] Specifically, when a startup operation of a multimedia application is detected, the terminal can start the multimedia application, display a multimedia interface through the multimedia application, and play multimedia content including a target object through the multimedia interface. Among them, the multimedia application can be a parent application or a child application. A child application is an application that can run in the operating environment provided by the parent application, and the parent application is a native application that runs on the operating system and provides a running environment for the child application. The parent application provides an environment for the operation of the child application and creates a logic layer processing unit and a view layer processing unit for implementing the child application. The view layer processing unit is used to process the view of the child application; the logic layer processing unit is used to process the interaction logic of the child application. The processing unit can specifically be a process or a thread, such as a logic layer thread and a corresponding view layer thread. The logic layer processing unit can run in a virtual machine, such as a JavaScript virtual machine.

[0099] In one embodiment, there can be multiple ways to start a multimedia application. For example, when the multimedia application is a sub-application in a parent application, the client of the parent application can determine the triggering of the sub-application startup operation by detecting the user clicking the sub-application icon on the screen; or by recognizing the user's voice information, determine the triggering of the sub-application startup operation; or by detecting the user's gesture, when the gesture matches the preset gesture, determine the triggering of the sub-application startup operation, but the present invention is not limited thereto.

[0100] In one embodiment, the target object displayed in the multimedia playback interface can be a default object or an object selected by the user. This embodiment is not limited here. For example, Figure 3 , R&D personnel can pre-designate specific objects in a video as target objects 302. For another example, a multimedia playback interface may have a target object selection control. When a multimedia application displays the multimedia playback interface, a user can use the target object selection control to designate one or more objects in the multimedia playback interface as target objects. A user is a person viewing the multimedia content played on the multimedia playback interface. Figure 3 A schematic diagram of a target object in one embodiment is shown.

[0101] Step S204: displaying an interactive entrance in the multimedia playback interface.

[0102] Among them, the interactive entrance refers to the entrance that triggers the interaction with the target object through virtual props.

[0103] Specifically, a multimedia application can display an interactive entry in the multimedia playback interface. The multimedia application can display the interactive entry when the multimedia content reaches a preset position; it can also display the interactive entry continuously from the start of multimedia content playback to the end of multimedia content playback; or it can display the interactive entry randomly during the multimedia content playback process. This embodiment is not limited here.

[0104] In one embodiment, reference Figure 3 The multimedia application may display an interactive entry 304 at a preset location on the multimedia playback interface. The interactive entry includes an interactive space entry control and an interactive entry cancellation control. A user can enter the interactive space by clicking the interactive space entry control, thereby interacting with the target object in the interactive space. The user can also cancel the interactive entry by clicking the interactive entry cancellation control, thereby not interacting with the target object.

[0105] In one embodiment, if the user does not trigger the interactive portal within a preset time period, the multimedia application will cancel the display of the interactive portal. For example, if the user does not click the interactive portal within 5 seconds, it can be assumed that the user does not want to interact with the target object at this time, and the multimedia application will cancel the display of the interactive portal.

[0106] Step S206 , in response to the triggering operation of the interactive entrance, an interactive animation is displayed in the area corresponding to the target object in the multimedia playback interface, and the interactive animation includes a virtual prop that interacts in an interactive manner that matches the facial expression of the target object.

[0107] Triggering actions refer to actions that trigger corresponding functions in various ways. For example, triggering actions can be touch, voice, and gesture. Touch actions include pressing, touching, sliding, and pressing and sliding. Interactive animation refers to the animation used to represent the transfer of virtual props between the user and the target object.

[0108] Specifically, the user can interact with the target object by triggering the interactive entrance. When the multimedia application determines that the user has triggered the interactive entrance, the multimedia application can respond to the trigger operation, obtain the current video stream to be played, and identify the video frames in the current video stream to be played to determine the area where the target object is located in the video frame, and identify the facial expression of the target object, and determine the target interaction method for interacting with the target object based on the facial expression of the target object. Furthermore, the multimedia application plays the interactive animation of the virtual props while playing the video stream. The interactive animation is displayed in the area corresponding to the target object in the multimedia playback interface, and the interactive animation includes virtual props that interact according to the target interaction method.

[0109] In one embodiment, the method for generating the above-mentioned recognition result library also includes: obtaining multimedia content played by the multimedia application, and performing frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain multiple extracted target video frames; for each target video frame, identifying the target object in the current target video frame to obtain a recognition result; wherein the recognition result includes the area where the facial part of the target object is located, and the facial expression of the target object; and storing the recognition results corresponding to each target video frame to obtain a recognition result library.

[0110] Specifically, a multimedia application may include a built-in facial recognition plug-in that extracts frames from multimedia content at a preset frame extraction frequency to obtain multiple target video frames. The plug-in then performs frame-by-frame facial recognition on the extracted multiple target video frames, obtaining a recognition result corresponding to each target video frame, and storing the recognition result for each target video frame in a recognition result library. The recognition result includes the target object's area and the target object's facial expression.

[0111] In one embodiment, a facial recognition module may also be provided in the server, and the facial recognition module is used to perform frame-by-frame facial recognition on the video played by the multimedia application to obtain a recognition result library.

[0112] In one embodiment, the target part includes a facial part, and determining the area where the target part of the target object played in the multimedia playback interface is located, as well as the facial expression of the target object, includes: determining the playback progress of the multimedia content played in the multimedia playback interface; filtering out the target recognition result from a preset recognition result library according to the playback progress, and determining the area where the facial part of the target object is located at the current moment, as well as determining the facial expression of the target object, based on the target recognition result.

[0113] Specifically, when a multimedia application determines that a user has triggered an interactive entry, the interactive animation drawing engine in the multimedia application can obtain the playback progress of the current video in real time, query the corresponding recognition results from the recognition result library based on the playback progress, and draw the corresponding interactive animation based on the queried recognition results. For example, the interactive animation drawing engine can obtain the millisecond-level playback progress of the multimedia application in real time, and query the recognition results of the video frame to be played in the current millisecond based on the playback progress. Based on the recognition results, the display area of ​​the interactive animation and the interactive method of the virtual props are determined. Based on the determined display area and interactive method, the video frame to be played in the current millisecond is displayed in the multimedia playback interface, and the interactive animation is drawn in the video frame to be played in the current millisecond.

[0114] In one embodiment, a facial recognition module is deployed on a server. The server performs frame-by-frame facial recognition on a video played by a multimedia application, generating a recognition result library. The server can then send the recognition result library to a terminal, allowing the terminal to draw an interactive animation based on the received recognition result library. The server can directly send the entire recognition result library to the terminal all at once, or it can send a portion of the recognition result library to the terminal upon request.

[0115] In the above embodiment, by pre-generating a facial recognition library, the location of the target object under the current playback progress and the facial expression of the target object can be quickly obtained based on the pre-generated facial recognition library, thereby improving the drawing efficiency of the interactive animation based on the quickly determined location information and facial expressions.

[0116] In one embodiment, the multimedia content is subjected to frame extraction processing according to a preset frame extraction frequency to obtain extracted multi-frame target video frames, including: performing frame extraction processing on the multimedia content according to the preset frame extraction frequency to obtain extracted video frames; performing a frame missing check on the extracted video frames according to the frame extraction frequency to obtain a check result, and performing supplementary frame extraction processing on the multimedia content according to the check result to obtain supplementary video frames; and obtaining multi-frame target video frames based on the extracted video frames and the supplementary video frames.

[0117] Specifically, to ensure the integrity of the extracted target video frames, when the multimedia content is subjected to frame extraction processing according to a preset frame extraction frequency to obtain extracted video frames, it is also necessary to perform a frame missing check on the extracted video frames according to the frame extraction frequency to obtain a test result. For example, when the frame extraction frequency determines that the first, third, and fifth frames should be extracted, and the extracted video frames are actually the first and fifth frames, it can be determined that the third frame is a missing video frame. At this time, the terminal performs supplementary frame extraction processing on the multimedia content based on the test result to obtain a supplementary video frame. For example, in the above example, the third frame is the supplementary video frame. When the extracted video frames and the supplementary video frames are obtained, the terminal can use the extracted video frames and the supplementary video frames as target video frames to identify the target object in each target video frame to obtain a recognition result.

[0118] In one embodiment, when an extracted video frame is extracted, the terminal may identify the extracted video frame to obtain a recognition result of the extracted video frame. The terminal traverses the recognition results of the extracted video frame to determine whether there are missing recognition results based on the frame extraction frequency. When it is determined that there are missing recognition results, the terminal extracts a supplementary video frame from the multimedia content based on the playback progress of the missing recognition result. The terminal identifies the supplementary video frame to obtain a recognition result of the supplementary video frame, and stores both the recognition result of the supplementary video frame and the recognition result of the extracted video frame in a recognition result library. It is easy to understand that this step can also be performed in the server.

[0119] In the above embodiment, by performing supplementary frame extraction processing on the multimedia content, the extracted target video frames can be made more complete, thereby making the recognition result library determined based on the more complete target video frames more complete.

[0120] In one embodiment, the recognition results corresponding to each target video frame are stored to obtain a recognition result library, including: determining the playback progress corresponding to each target video frame, and dividing the playback progress of the target video frame into multiple playback progress intervals according to a preset time window and a time window movement step; corresponding to each playback progress interval, determining the difference between the recognition results of each target video frame falling within the current playback progress interval, and merging the recognition results whose differences are less than a difference threshold; and storing the merged recognition results to obtain a recognition result library.

[0121] Specifically, in order to reduce the amount of data of the recognition results stored in the recognition result library, similar recognition results can be merged. For example, if the error between the recognition results of adjacent frames in a continuous video segment is less than a certain range, the recognition results in the continuous video segment are merged to reduce the amount of data.

[0122] The terminal determines the playback progress corresponding to each target video frame and determines a preset time window and time window movement step size. Based on the preset time window and time window movement step size, the playback progress of the target video frame is divided into multiple playback progress intervals. For example, when the time window is 2 seconds and the time window movement step size is 2 seconds, 0 minutes 0 seconds 0 milliseconds to 0 hours 0 minutes 2 seconds 0 milliseconds can be divided into one playback progress interval, and 0 minutes 2 seconds 0 milliseconds to 0 hours 0 minutes 4 seconds 0 milliseconds can be divided into another playback progress interval, and so on. For each playback progress interval, the terminal determines the difference between the recognition results of each target video frame falling within the current playback progress interval and merges the recognition results whose differences are less than a difference threshold. The terminal stores the merged recognition results to obtain a recognition result library. It is easy to understand that the step of merging the recognition results can also be completed on the server, so that the server can generate a smaller recognition result library. When the terminal requests to obtain the recognition result library, the request time of the recognition result library can be reduced, thereby improving the transmission efficiency of the recognition result library.

[0123] In this embodiment, by merging similar recognition results, the number of recognition results stored in the recognition result library can be reduced, thereby reducing the probability of data redundancy.

[0124] In one embodiment, an interactive animation is displayed within an area corresponding to a target object in a multimedia playback interface, including: interpolating recognition results in a recognition result library according to a frame extraction frequency and a frame rate of multimedia content played on the multimedia playback interface to obtain an interpolation result; determining a correspondence between position information of the target object in the multimedia content and a playback progress according to the interpolation result; determining a trajectory curve of a virtual prop according to the correspondence, and performing smooth optimization processing on the trajectory curve to obtain an optimized trajectory curve; and displaying an interactive animation within an area corresponding to the target object in the multimedia playback interface, wherein the virtual prop in the interactive animation moves according to the optimized trajectory curve.

[0125] Specifically, because the video frames corresponding to the recognition results are extracted at 25 frames per second, while the frame rate of mobile devices is generally above 60, to achieve a smooth animation effect, the multimedia application can smooth the recognition results to ensure a smooth real-time animation trajectory. The multimedia application uses the Catmull-Rom algorithm (a curve drawing algorithm) to interpolate the recognition results in the recognition result library to obtain an interpolated result, which improves the visual transmission frame rate per second. Based on the interpolated result, the terminal determines the correspondence between the target object and the playback progress during the time period from the beginning to the end of the interactive space. Based on this correspondence, the terminal determines the trajectory curve of the virtual prop. The terminal uses the Kalman filter function (an algorithm that uses the linear system state equation to optimally estimate the system state based on the system input and output observation data) to smooth and optimize the trajectory curve, resolve small data glitches, and avoid slight jitter in the interactive animation. Finally, the multimedia application draws the interactive animation and calculates the coordinates, width, height, animation transformation angle, and other styles of the virtual props relative to the character in each frame based on the real-time coordinates and size of the target character.

[0126] In the above embodiment, the probability of slight jitter in the interactive animation can be reduced by smoothing the trajectory curve.

[0127] In one embodiment, the process of drawing an interactive animation on a terminal may include data preprocessing, data smoothing, and animation drawing. Data preprocessing involves preliminary cleaning of the recognition results in the recognition result library before animation drawing. Recognition results in the recognition result library often have multiple similarities, occasional anomalies, and redundancy, making them unsuitable for direct use on the front end. Therefore, pre-optimization of the recognition results is necessary. The terminal determines whether the playback progress corresponding to the recognition result aligns with the frame extraction frequency. If not, frame insertion is performed. Furthermore, the terminal can set multiple time intervals, each of which can store recognition results for adjacent video frames, with the difference between each recognition result being less than a preset difference threshold. Since the array stores recognition results for adjacent video frames, the terminal can determine the start and end times of the time interval based on the playback times corresponding to the adjacent video frames, thereby generating an optimized recognition result library. Furthermore, in response to a multimedia application's request for recognition results, the facial recognition plug-in can return the target recognition results required by the multimedia application from the optimized recognition result library, without returning unnecessary recognition results to the front end.

[0128] Furthermore, since the video frames corresponding to the recognition results are extracted at 25 frames per second, while the frame rate of mobile devices is generally above 60, in order to achieve a smooth animation effect, the multimedia application can perform data smoothing on the optimized recognition results to make the real-time animation trajectory smooth. The multimedia application uses the Catmull-Rom algorithm (a curve drawing algorithm) to interpolate and fill frames on the pre-processed data to improve the visual fps, and uses the Kalman filter function (an algorithm that uses linear system state equations and system input and output observation data to optimally estimate the system state) to smooth and optimize the interpolated trajectory curve to resolve small-scale data glitches and avoid slight jitter in the animation. Finally, the multimedia application draws the interactive animation and calculates the coordinates, width and height, animation transformation angle, and other styles of the virtual props relative to the character in each frame based on the real-time coordinates and size of the target character.

[0129] In the above embodiment, by pre-generating the recognition results, the interactive animation can be drawn directly based on the pre-generated recognition results, thereby improving the efficiency of the interactive animation drawing. By optimizing the recognition results, the efficiency and effect of the interactive animation drawing can be improved based on the optimized recognition results.

[0130] In one embodiment, a user can enter an interactive space by triggering an interactive portal and interact with a target object in the interactive space using virtual props. For example, a user can gift a virtual prop to a target object in the interactive space, and the multimedia application can then display an interactive animation based on the user's gifting action. When a user enters the interactive space, the multimedia application can determine the cumulative time the user has been in the interactive space and close the interactive space when the cumulative time reaches a preset value. For example, the multimedia application can start a timer when the user enters the interactive space and close the interactive space when it determines that the user has been in the interactive space for one minute.

[0131] The target period is defined as the time between the user entering the interactive space and the space closing. The interactive animation rendering engine queries a pre-set recognition result library for the recognition results of each video frame within the target period. Based on the queried recognition results, the interactive animation display area and the interactive methods of the virtual props are updated in real time. For example, during the target period, the interactive animation display area moves with the movement of the target object, and the interactive methods of the virtual props in the interactive animation change with the target object's expression.

[0132] In a real-time example, a multimedia application pre-sets a correspondence between facial expressions and interaction methods. When a target object's facial expression is acquired, the multimedia application can determine the target interaction method corresponding to the target object's facial expression based on this correspondence, thereby displaying a virtual prop that interacts according to the target interaction method in an interactive animation. For example, the virtual prop can be a virtual hammer. When the target object's facial expression is determined to be happy, the multimedia application determines the corresponding target interaction method to be spinning in place, thereby displaying an interactive animation of a virtual hammer spinning in place in the area where the target object's target part is located. When the target object's facial expression is determined to be sad, the multimedia application determines the corresponding target interaction method to be hitting the target part, thereby displaying an interactive animation of a virtual hammer hitting the target part in the area where the target object's target part is located.

[0133] In one embodiment, when multiple target objects are displayed in the multimedia playback interface, the multimedia playback interface displays an interactive animation in the area corresponding to each target object. Alternatively, when multiple target objects are displayed in the multimedia playback interface, the multimedia playback interface displays an interactive animation in the area corresponding to a random target object.

[0134] In one embodiment, the prop shape of the virtual props and the appearance timing of the interactive entrance can be configured in advance, so that the multimedia application can display the corresponding interactive entrance and virtual props according to the configuration information. In the above-mentioned virtual prop interaction method, by displaying the multimedia playback interface, the target object and the interactive entrance can be displayed through the multimedia playback interface. By displaying the interactive entrance, in response to the triggering operation of the interactive entrance, an interactive animation of the virtual props that interacts in an interactive manner that matches the facial expression of the target object is displayed within a range corresponding to the target object. Since an interactive animation of the virtual props that interacts in an interactive manner that matches the facial expression of the target object can be displayed within a range corresponding to the target object in the multimedia playback interface, compared to the traditional way of participating in the interaction only by sending barrages, the present application can not only establish a strong connection with the target object, but also enhance the diversity of the interaction methods with the target object in the multimedia content.

[0135] Furthermore, the present invention can display interactive portals while the user is watching multimedia content, allowing the user to conveniently interact with the target object based on the displayed interactive portal. This not only improves the efficiency of interaction but also enables the user to experience a good emotional resonance, greatly enhancing the effectiveness of the interaction. Furthermore, since the user only needs to trigger the displayed interactive portal to interact with the target object through virtual props, the efficiency of human-computer interaction is greatly improved.

[0136] In one embodiment, displaying the interactive entrance in the multimedia playback interface includes: when it is determined that the multimedia content in the multimedia playback interface is played to a designated position, displaying the interactive entrance in the multimedia playback interface.

[0137] Specifically, when multimedia content is played through a multimedia application, the multimedia application can determine the playing progress of the multimedia content in real time, and when it is determined that the multimedia content in the multimedia playback interface has been played to a specified position, an interactive entrance is displayed in the multimedia playback interface. Figure 3 When the multimedia content is played to the designated position, the multimedia application can display the interactive entrance 304 at a preset position on the multimedia playback interface. The designated position can be freely set according to needs. For example, the plot turning point in the video can be set as the designated position, and the exciting part of the video can be set as the designated position.

[0138] In one embodiment, the developer may configure the designated location in advance, so that when the multimedia content is played to the designated location, the multimedia application may display an interactive entrance in the multimedia playback interface.

[0139] In the above embodiment, by displaying an interactive entrance when the multimedia content is played to a specified position, the user can interact with the target object in the specified plot, thereby improving the interactive effect.

[0140] In one embodiment, in response to a triggering operation on an interactive entrance, an interactive animation is displayed within an area corresponding to a target object in a multimedia playback interface, including: in response to a triggering operation on an interactive entrance, entering an interactive space, and displaying interactive controls of virtual props in the interactive space; in response to a specified operation on an interactive control, displaying an interactive animation within an area corresponding to a target object in a multimedia playback interface.

[0141] Specifically, when a user triggers an interactive entry, the multimedia application can respond to the trigger and allow the user to enter the interactive space. The interactive space refers to the period of time during which the user can interact with the target object. Furthermore, the multimedia application displays interactive controls within the multimedia playback interface within the interactive space, and in response to the user's designated actions on the interactive controls, displays interactive animations within the area of ​​the multimedia playback interface corresponding to the target object.

[0142] For example, reference Figure 4When the user clicks on the interactive entrance to enter the interactive space, the multimedia application can display the interactive control 402 in the multimedia playback interface, so that the user can trigger the multimedia application to display the interactive animation by clicking on the interactive control 402. That is, the user can give the virtual props to the target object by clicking on the interactive control 402, so that the multimedia application displays the interactive animation including the given virtual props. Figure 4 A schematic diagram of an interactive control in one embodiment is shown.

[0143] In this embodiment, the interactive control can be displayed in response to a triggering operation on the interactive portal, so that the interactive animation can be displayed only by responding to a specified operation on the interactive control.

[0144] In one embodiment, an interactive animation is displayed within an area corresponding to a target object in a multimedia playback interface, including: determining an area where a target part of the target object played in the multimedia playback interface is located, and a facial expression of the target object; obtaining a target interaction method that matches the facial expression of the target object; and displaying an interactive animation including a virtual prop that interacts according to the target interaction method within an area corresponding to the area where the target part is located.

[0145] Specifically, during video playback by a multimedia application, the multimedia application can determine the area in the video stream where the target part of the target object is located, as well as the target object's facial expression. The target part can be freely set as needed; for example, the target part can be the target object's face, hand, or upper torso. Furthermore, the multimedia application determines a target interaction method that matches the target object's facial expression and displays an interactive animation including virtual props that interact according to the target interaction method within a region corresponding to the area where the target part is located. The region corresponding to the area where the target part is located can be freely set as needed. For example, when the target part is the target object's face, the region used to display the interactive animation can be the region adjacent to and surrounding the facial region. For another example, when the target part is the target object's upper torso, the region used to display the interactive animation can be the region adjacent to and surrounding the upper torso.

[0146] In one embodiment, the multimedia application may identify each video frame in the video stream using a recognition algorithm before displaying the interactive entrance, obtain a recognition result, and store the recognition result in correspondence with the playback progress of the video frame in the video stream. The multimedia application may also identify the video frames in the video stream in real time in response to the triggering operation of the interactive entrance to obtain a recognition result. The recognition result includes the area where the target part of the target object in the video frame is located, and the facial expression of the target object. The recognition algorithm can be freely set according to needs. For example, the recognition algorithm can be a pre-trained machine learning model, or it can be an expression recognition algorithm that comes with the iOS or Android system.

[0147] When a multimedia application pre-recognizes video frames and stores the recognition results, in response to a triggering operation on an interactive portal, the multimedia application can filter the target recognition results from the stored results in real time and adjust the animation content and display area of ​​the interactive animation based on the target recognition results. When a multimedia application pre-recognizes video frames in real time, the multimedia application can determine the animation content and display area of ​​the interactive animation based on the real-time recognition results.

[0148] In the above embodiment, by determining the area where the target part of the target object is located and the facial expression of the target object, the display area and animation content of the interactive animation can be adjusted in real time according to the area where the target part is located and the facial expression, thereby not only increasing the interactivity with the target object, but also enhancing the interactive effect with the target object.

[0149] In one embodiment, the target part includes a facial part, and determining the area where the target part of the target object played in the multimedia playback interface is located, as well as the facial expression of the target object, includes: obtaining facial features of the target object; wherein the facial features are extracted from a facial image collected for the target object; determining facial key points of the target object based on the facial features; and determining the area where the facial part of the target object is located, and determining the facial expression of the target object based on the facial key points.

[0150] Specifically, the target part can be a facial part. When it is necessary to determine the area where the facial part of the target object is located and the facial expression of the target object, the multimedia application can obtain the pre-set facial features of the target object and determine the facial key points of the target object based on the facial features, thereby obtaining the area where the facial part is located and the facial expression based on the facial key points.

[0151] In one embodiment, the background server of the multimedia application may pre-collect facial images of the target object, extract facial features from the facial images, and send the facial features to a recognition algorithm so that the recognition algorithm can filter out the target object from multiple objects in the video frame based on the facial features and determine the target object's facial area and facial expression.

[0152] In one embodiment, the background server of the multimedia application may pre-collect facial images of each object in the video, so that when the user selects a target object through the target object selection control in the multimedia playback interface, the background server may filter out the facial images corresponding to the target object based on the user's selection operation, extract facial features from the facial images corresponding to the target object, and send the facial features to the recognition algorithm, so that the recognition algorithm can determine the target object in the video frame based on the facial features and recognize the facial expression of the target object.

[0153] In one embodiment, reference Figure 5 , Figure 5 A schematic diagram of facial landmarks in one embodiment is shown. The recognition algorithm can locate the facial features of a target subject in a video frame to determine a preset number of facial landmarks that form a facial outline. These facial landmarks include eyebrow landmarks, eye landmarks, nose landmarks, mouth landmarks, facial outline landmarks, and eyeball landmarks.

[0154] In one embodiment, reference Figure 6 When the facial key points are obtained, the recognition algorithm can determine the target object's facial expression based on the facial key points. Facial expressions include neutral, happy, surprised, angry, sad, disgusted, etc. Figure 6 A schematic diagram of facial expressions in one embodiment is shown.

[0155] In the above embodiment, by obtaining facial features, the target object that needs to interact can be accurately determined based on the facial features; by determining the facial key points of the target object, the area where the facial parts of the target object are located and the facial expression of the target object can be determined based on the facial key points. In this way, an interactive animation with an interactive effect can be obtained based on the area where the facial parts are located and the facial expression.

[0156] In one embodiment, the interaction mode includes the display effect, swing amplitude and swing frequency of the virtual prop. The above method also includes: when the facial expression of the target object belongs to the first expression category, determining the target interaction mode that matches the facial expression of the target object as the first interaction mode; when the facial expression of the target object belongs to the second expression category, determining the target interaction mode that matches the facial expression of the target object as the second interaction mode; wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual prop in the first interaction mode is different from that of the virtual prop in the second interaction mode.

[0157] Specifically, the multimedia application may pre-store the correspondence between facial expression categories and interaction methods. When a target object's facial expression is recognized, the multimedia application may determine the expression category to which the facial expression belongs. If the facial expression is determined to belong to the first expression category, the target interaction method matching the target object's facial expression is determined to be the first interaction method; if the facial expression is determined to belong to the second category, the target interaction method matching the target object's facial expression is determined to be the second interaction method. The facial categories can be freely set as needed. For example, a neutral expression can be classified as the first expression category, and happiness, surprise, anger, sadness, and disgust can be classified as the second expression category. The interaction methods corresponding to different expression categories can also be freely set as needed. For example, the interaction methods may include the display effects, swing amplitude, and swing frequency of virtual props. Display effects include luminous effects and particle display effects.

[0158] In one embodiment, reference Figure 7 , developers can pre-set the first interactive mode as follows: virtual prop 702 does not emit light, luminous particles are not displayed around virtual prop 702, and the virtual prop 702 swings with an amplitude of a degree and a frequency of b seconds per swing; and set the second interactive mode as virtual prop 704 emits light, luminous particles are displayed around virtual prop 704, and the virtual prop 704 swings with an amplitude of c degrees and a frequency of d seconds per swing. Thus, when it is determined that the target object's facial expression belongs to the first expression category, the multimedia application can display an interactive animation within the area corresponding to the target object, wherein the virtual prop in the interactive animation is a virtual prop that does not emit light and has no luminous particles, and swings back and forth with an amplitude of a degree and a frequency of b seconds per swing, thereby achieving a small tapping effect based on the back-and-forth swing. When it is determined that the facial expression of the target object belongs to the second expression category, the multimedia application can display an interactive animation within the area corresponding to the target object, wherein the virtual props in the interactive animation are luminous virtual props surrounded by luminous particles, and the virtual props swing back and forth with a swing amplitude of c degrees and a swing frequency of d seconds / round, so as to achieve a large-scale knocking effect based on the back-and-forth swinging. Figure 7 A schematic diagram of an interactive mode of a virtual prop in an embodiment is shown.

[0159] In one embodiment, when it is determined that the facial expression of the target object belongs to the second expression category, a virtual prop may be displayed in the interactive animation, wherein the virtual prop includes luminous particles.

[0160] In one embodiment, the shape of the virtual prop can be freely set as needed. For example, the virtual prop can be set as a virtual hammer, a virtual person, a virtual animal, or a virtual plant. When the virtual prop is a virtual hammer, the first interaction method can be the virtual hammer hitting the target object with a small amplitude near the target object's face; the second interaction method can be the virtual hammer hitting the target object with a large amplitude near the target object's face.

[0161] In one embodiment, the interaction mode of the virtual prop can be freely set according to needs. For example, the virtual prop can interact with the target object by rotating, or cling to the target part of the target object. Therefore, the interaction mode can also include rotation speed, movement speed, etc.

[0162] In one embodiment, reference Figure 8 , Figure 8 A flow chart of determining an interaction method in one embodiment is shown. S802, the terminal may collect facial images of the target object, S804 and send the collected facial images to the server, S806 so that the server determines the target object to interact with the user based on the received facial images. S808 The server determines the facial features of the target object and sends the facial features to the recognition plug-in, S810 The recognition plug-in may recognize the target object in the video frame based on the received facial features to obtain a recognition result. S812 The recognition plug-in returns the recognition result to the multimedia application, S814 The multimedia application in the terminal determines the category to which the facial expression of the target object belongs based on the received recognition result, and determines whether the facial expression of the target object belongs to the first expression category, S816 When the facial expression belongs to the first expression category, the interaction method of the virtual prop is determined to be the first interaction method, S818 When the facial expression belongs to the second expression category, the interaction method of the virtual prop is determined to be the second interaction method.

[0163] In the above embodiment, by determining the facial expression of the target object, the corresponding interaction method can be determined based on the facial expression, so that the virtual props can interact according to the determined interaction method. This not only greatly enriches the diversity of interaction methods, but also improves the correlation between the interaction content and the specific behavior of the target object.

[0164] In one embodiment, in response to a triggering operation on an interactive entrance, an interactive animation is displayed within an area corresponding to a target object in a multimedia playback interface, including: in response to a triggering operation on the interactive entrance, an interactive animation is displayed in which a virtual prop appears from the interactive entrance, moves toward the area where the facial part of the target object is located, and interacts.

[0165] Specifically, the multimedia application can display an interactive animation in the area corresponding to the target object in response to a user triggering an interactive portal. The interactive animation includes a virtual prop that appears from the interactive portal and moves toward the target object's facial area, interacting in a targeted interactive manner corresponding to the facial expression during the movement.

[0166] In one embodiment, the multimedia application may display the interactive control in response to the user's triggering operation on the interactive portal, and display the interactive animation in the area corresponding to the target object in response to the specified operation on the interactive control. Figure 9 In response to the user's designated operation on the interactive control 902, the multimedia application can present a virtual prop to the target object. At this time, after the virtual prop 904 appears from the interactive control 902, it moves toward the area where the facial area of ​​the target object 906 is located, and interacts in the target interaction method corresponding to the facial expression during the movement, for example, hitting the target object 906 with a small amplitude during the movement. Figure 9 A schematic diagram showing the movement principle of a virtual prop in one embodiment is shown.

[0167] In one embodiment, the multimedia playback interface may display a custom control for the target part and a custom control for the motion path. The user may use the custom control to specify the target part of the target object, and the custom control to specify the motion path of the virtual prop. For example, the user may define the target part as the face, and define the motion path as the virtual prop moving toward the target object's facial area after the interactive control appears, and then moving clockwise around the target part of the target object.

[0168] In the above embodiment, by displaying a virtual prop that moves toward the facial area of ​​the target object, the user can interact with the target object based on the virtual prop, thereby enhancing the relevance between the user and the target object in the video content.

[0169] In one embodiment, there are multiple virtual props. After the multiple virtual props move to the area where the facial parts of the target object are located, they surround the facial parts of the target object to interact, and each virtual prop interacts in an interactive manner that matches the facial expression of the target object.

[0170] Specifically, there can be multiple virtual props in the interactive animation. Multiple virtual props can appear from the interaction entrance in turn, move to the area where the target object's facial parts are located, and then surround the target object's facial parts. Each virtual prop interacts in an interactive manner that matches the target object's facial expression.

[0171] For example, reference Figure 10 ,When the virtual prop is a virtual hammer, multiple virtual hammers can appear from the ,interaction entrance in sequence and surround the facial part of the target object, ,e.g. Figure 10 The middle dashed path surrounds the target object's facial area, and each virtual hammer hits the target object's facial area with a small amplitude. Figure 10 A schematic diagram illustrating the interaction of virtual props in one embodiment is shown. In one embodiment, a multimedia application can display an interactive animation including multiple virtual props in the area of ​​a target object's face in response to a user's designated action on an interactive control. For example, a user only needs to click the interactive control once to cause the multimedia application to display multiple virtual props. The multimedia application can also determine the number of virtual props included in the interactive animation based on the number of times the user performs designated actions on the interactive control, and display a corresponding number of virtual props in the interactive animation. For example, if the user clicks the interactive control 15 times, the multimedia application will display 15 virtual props.

[0172] In the above embodiment, by displaying multiple virtual props and surrounding the facial parts of the target object for interaction, the diversity of interaction can be greatly increased, thereby improving the user experience.

[0173] In one embodiment, in response to a specified operation on an interactive control, an interactive animation is displayed in an area corresponding to a target object in a multimedia playback interface, including: determining all login accounts currently entering the interactive space, and determining the first number of operations for all login accounts to trigger the specified operation on the interactive control; wherein the login account is an account for viewing the multimedia content played on the multimedia playback interface; the interactive animation is displayed in an area corresponding to the target object in the multimedia playback interface; wherein the number of virtual props included in the interactive animation is updated in real time as the first number of operations accumulates.

[0174] Specifically, because multiple users may enter the same interactive space during the same time period, trigger interactive controls together, and gift virtual props to the target object together, the multimedia application can determine all logged-in accounts currently entering the interactive space. A logged-in account refers to the account of the user viewing the multimedia content played on the multimedia playback interface. For example, when user A watches video a through the multimedia application and enters the interactive space at 10:00, and user B also watches video a through the multimedia application and enters the interactive space at 10:00, the multimedia application can determine that the total number of logged-in accounts currently entering the interactive space is A and B, and that the total number of logged-in accounts currently entering the interactive space is 2.

[0175] Furthermore, the multimedia application can count the number of first operations that all logged-in accounts trigger for a specified operation on the interactive control within the statistical period from the time the logged-in account enters the interactive space to the current moment, and determine the number of virtual props included in the interactive animation based on the first number of operations. For example, in the above example, when the current time is 10:05, the multimedia application can count the number of times user A and user B click on the interactive control together from 10:00 to 10:05, and use this number as the first number of operations, so that the number of virtual props in the interactive animation displayed at the current moment can be determined based on the first number of operations. Among them, the specific method of determining the number of virtual props in the interactive animation based on the first number of operations can be freely set according to needs. For example, the number of virtual props can be determined by the following formula: number of virtual props = first number of operations / 3.

[0176] It is worth noting that the multimedia application can count in real time the number of first operations of the specified operation triggered by all logged-in accounts during the statistical period from the time the logged-in account enters the interactive space to the current moment, thereby updating the number of virtual props in the interactive animation in real time based on the real-time count of the number of first operations, that is, the number of virtual props included in the interactive animation is updated in real time as the number of first operations accumulates.

[0177] In this embodiment, since the number of virtual props included in the interactive animation is updated in real time as the number of first operations accumulates, any user entering the interactive space can view the virtual props given by others, thereby enhancing the user's sense of joint participation.

[0178] In one embodiment, an interactive animation is displayed within an area corresponding to a target object in a multimedia playback interface, including: determining the number of virtual props displayed in the interactive animation and the display color of the virtual props based on the first operation number; and displaying the interactive animation within an area corresponding to the target object in the multimedia playback interface based on the display color and the number of virtual props.

[0179] Specifically, the number of virtual props and the color of the virtual props in the interactive animation may change with the change of the first operation number. When the first operation number is obtained, the multimedia application may determine the number of virtual props based on the first operation number, for example, by dividing the first operation number by 3 to obtain the number of virtual props. In addition, the multimedia application may also determine the display color of the virtual props based on the first operation number, for example, based on the correspondence between the first operation number and the display color. Thus, the multimedia application may display the interactive animation including the virtual props with the number of props within the area corresponding to the target object in the multimedia playback interface, wherein the color of the virtual props in the interactive animation is the display color determined based on the first operation number.

[0180] In one embodiment, since the display area of ​​an interactive animation is limited, the number of virtual props in the interactive animation can be limited to a reasonable range to reduce the probability of virtual props overlapping in the interactive animation and improve the user's viewing experience. For example, the maximum number of virtual props displayed can be set to 15. When the number of virtual props in the interactive animation reaches 15 based on the first number of operations, the number of virtual props in the interactive animation will not be increased.

[0181] When the number of virtual props in the interactive animation is no longer increased, the multimedia application can represent the increase in the number of props by changing the display color of the virtual props. Exemplarily, the multimedia application can use the moment when the number of virtual props in the interactive animation reaches the maximum display number as the benchmark historical moment, and change the display color of the virtual props when the difference between the first number of operations at the current moment and the first number of operations at the benchmark historical moment reaches a preset difference threshold. Further, the multimedia application uses the current moment as the benchmark historical moment, and returns to the moment when the difference between the first number of operations at the current moment and the first number of operations at the benchmark historical moment reaches a preset difference threshold, and the step of changing the prop color of the virtual props continues to be executed until the interactive space ends. For example, referring to Figure 11 The multimedia application can change the display color of the virtual item every time the first operation count increases by 12 times. For example, when the first operation count is less than or equal to 15, the color of the virtual item is golden yellow. When the first operation count is greater than 15 and less than or equal to 27, the color of the virtual item changes to orange. When the first operation count is greater than 27 and less than or equal to 39, the color of the virtual item changes to red. Figure 11 A schematic diagram showing the change in the display color of a virtual prop in one embodiment is shown.

[0182] In the above embodiment, since the number and color of virtual props in the interactive animation can change with the number of first operations, the user's sense of participation can be greatly increased, thereby improving the interaction effect with the target object.

[0183] In one embodiment, after determining the first number of operations for all logged-in accounts to jointly trigger a specified operation for the interactive control, the above-mentioned virtual prop interaction method also includes: determining the number of accounts of all logged-in accounts that jointly trigger the specified operation for the interactive control; and displaying the number of accounts and the first number of operations at the first position corresponding to the interactive control.

[0184] Specifically, in order to enhance the user's sense of joint participation, the multimedia application can determine the number of all logged-in accounts that jointly trigger a specified operation for the interactive space, and determine the first operation count of all logged-in accounts that jointly trigger the interactive control, and display the number of accounts and the first operation count at the first position corresponding to the interaction. Figure 9 The multimedia application may display text including the number of accounts and the first number of operations at the first position 908: "A total of 100 people jointly gave away 200 virtual props", where "100" is the number of accounts and "200" is the first number of operations.

[0185] In the above embodiment, by displaying the number of accounts and the first operation times at the first position, the users' sense of joint participation can be greatly enhanced, thereby strengthening the relevance between multiple users.

[0186] In one embodiment, the above method also includes: determining a second operation number for the local login account to trigger a specified operation for the interactive control; and displaying the second operation number at a second position corresponding to the interactive control.

[0187] The local login account refers to the account logged into the terminal locally and used to view multimedia content played on the multimedia playback interface. The local login account can be one of the total login accounts. For example, when user A logs in to a multimedia application through the local terminal to view multimedia content, user A's login account is considered the local login account relative to the local terminal.

[0188] Specifically, when a user logs in to a multimedia application through a local login account, the user can trigger a specified operation for an interactive control through the local login account. At this time, the multimedia application can count the number of operations that the local login account has performed to trigger a specified operation for the interactive control from the time the user entered the interactive space to the current moment, and use the number of operations as the second number of operations. For example, a user can give a virtual prop to a target object by clicking on an interactive control, so that the multimedia application can count the number of times the user clicks on the interactive control to obtain the second number of operations. Furthermore, when the second number of operations is obtained, the multimedia application can display the second number of operations at a second position corresponding to the interactive control. For example, referring to Figure 10 , the multimedia application can display the second number of operations “A times” at the second position.

[0189] In one embodiment, the number of virtual props in the interactive animation and the display color of the virtual props can also be updated as the second operation count accumulates. For example, the multimedia application can divide the second operation count by 3 to obtain the number of virtual props in the interactive animation, and update the display color of the virtual props in the interactive animation in real time based on the correspondence between the second operation count and the display color.

[0190] In one embodiment, the multimedia application may further display the power value corresponding to a designated operation of the interactive control by the local account in response to the designated operation.

[0191] In the above embodiment, by displaying the second operation count at the second position, the user can intuitively understand the number of times the interactive control is triggered by the user, thereby improving the user experience.

[0192] In one embodiment, the above-mentioned virtual prop interaction method also includes: determining the storage quantity of virtual props included in the local login account; displaying the storage quantity at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the second operation number accumulates.

[0193] Specifically, a plurality of virtual props may be pre-stored in the local login account, so that the user can give the pre-selected stored virtual props to the target object to interact with the target object. When the user enters the interactive space, the multimedia application can determine the number of virtual props pre-stored in the local login account, obtain the storage quantity, and display the storage quantity at the third position corresponding to the interactive control. When the user triggers the interactive control to give the virtual props to the target object, the storage quantity displayed at the third position may decrease as the number of the user's second operation increases. For example, referring to Figure 12, the multimedia application can display the storage quantity "15" 1202 at the third position and update the storage quantity 1202 in real time according to the second operation number. For example, each time the second operation number increases, the actual storage quantity at the third position decreases by one. Figure 12 A schematic diagram of a multimedia playback interface in an embodiment is shown.

[0194] In one embodiment, reference Figure 12 When the logged-in account first enters the interactive space, the multimedia playback interface may not display any interactive animation. When the terminal determines that the logged-in account has gifted a virtual item to the target object, the virtual item may appear from the "Gift" space and move radially along a dotted line toward the target object's facial area. If the terminal determines that the logged-in account has gifted multiple virtual items to the target object, the multiple virtual items may appear sequentially from the interactive entrance, move to the target object's facial area, and then surround the target object's facial area.

[0195] In one embodiment, in order to enhance user enthusiasm for interactive participation, when a user enters an interactive space, the background server of the multimedia application may pre-store multiple virtual props in a local login account, so that the user can give the pre-stored virtual props to the target object.

[0196] In the above embodiment, by displaying the storage quantity at the third position, the user can intuitively understand the number of virtual props remaining in his or her account, thereby improving the efficiency of determining the number of virtual props remaining.

[0197] In one embodiment, after displaying the storage quantity at the third position corresponding to the interactive control, the above-mentioned virtual prop interaction method also includes: when the storage quantity is updated to zero as the second operation number accumulates, displaying the resource value required to replenish the storage quantity of the virtual prop, and providing a resource value replenishment entry; the resource value replenishment entry is used to implement the resource transfer operation.

[0198] Specifically, when the storage quantity of the virtual props included in the local login account is updated to zero as the second operation number accumulates, the multimedia application can display the resource value required to replenish the storage quantity of the virtual props. Figure 12 , the multimedia application may display the resource value “0.8 / piece” 1204 required to replenish the storage quantity of the virtual props at the third position, so that the user may choose whether to replenish the storage quantity of the virtual props based on the displayed information.

[0199] Furthermore, when the user chooses to replenish the storage quantity of virtual props, the multimedia application can determine the resource value required to replenish the virtual props and transfer the resource value from the local login account to the multimedia application account to achieve the replenishment of the virtual props. When the resource value of the local login account reaches zero, the multimedia application can display a resource value replenishment entry and, in response to the user triggering the resource value replenishment entry, display a resource value replenishment interface, allowing the user to transfer resource values ​​from a third account to the local login account through the resource value replenishment interface. The multimedia application account refers to the account corresponding to the multimedia application development team, and the third account refers to any other account associated with the user, excluding the local login account.

[0200] In this embodiment, by displaying the resource values ​​required to replenish the storage quantity of virtual props, users can independently choose whether to replenish virtual props based on the displayed information, thereby improving the user experience; by displaying the resource value replenishment interface, users can transfer resources based on the resource value replenishment interface, thereby realizing the replenishment of virtual props.

[0201] In one embodiment, the interactive control includes a first interactive control and a second interactive control. In response to a triggering operation on an interactive entrance, the interactive control enters the interactive space and displays the interactive control of the virtual prop in the interactive space, including: in response to a triggering operation on the interactive entrance, the interactive control enters the interactive space and displays the first interactive control in the interactive space; in response to a specified operation on the interactive control, the interactive animation is displayed in the area corresponding to the target object in the multimedia playback interface, including: in response to the first triggering operation on the first interactive control, the display of the first interactive control is canceled, and the second interactive control is displayed instead, and the interactive animation is displayed in the area corresponding to the target object in the multimedia playback interface.

[0202] Specifically, the interactive controls may include a first interactive control and a second interactive control. When the multimedia application enters the interactive space in response to a triggering operation of the interactive portal, the multimedia application may display the first interactive control in the multimedia playback interface. When the user triggers the first interactive control, the multimedia application may, in response to the first triggering operation on the first interactive control, replace the first interactive control with the second interactive control and display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

[0203] For example, reference Figure 13 When a user enters the interactive space, the multimedia application may display a first interactive control 1302 with the text "Gift". When the user clicks the first interactive control, the multimedia application may display a second interactive control 1304 with the text "Combo". This allows the user to give away virtual items multiple times through the change of the text. Figure 13A schematic diagram of an interactive control in one embodiment is shown.

[0204] In this embodiment, by displaying different interactive controls at different stages, the diversity of interaction can be improved.

[0205] In one embodiment, the above-mentioned virtual prop interaction method also includes: in response to a second trigger operation on the second interactive control, displaying an interactive countdown prompt animation at a fourth position corresponding to the second interactive control; when the interactive countdown prompt animation ends, canceling the display of the second interactive control and the interactive animation.

[0206] Specifically, the multimedia application can respond to the user's second trigger operation on the second interactive control and display an interactive countdown prompt animation at the fourth position corresponding to the second interactive control, thereby prompting the user to close the interactive space through the interactive countdown prompt animation. The fourth position can be freely set according to needs, for example, refer to Figure 14 A ring 1402 may be displayed around the second interactive control, and the colored area within the ring may decrease clockwise to indicate the remaining time until the interactive space is closed. When the colored area within the ring reaches 100%, the remaining time until the interactive space is closed is considered zero. At this point, the multimedia application displays a third control 1404 and cancels the display of the second interactive control and the interactive animation. Figure 14 A schematic diagram of an interactive countdown prompt animation in one embodiment is shown.

[0207] In one embodiment, the multimedia application may determine whether the user triggers a second triggering operation on the second interactive control within a preset time period after the second interactive control is displayed. If the user does not trigger the second interactive control within the preset time period, the multimedia application cancels the display of the second interactive control and switches to displaying the first interactive control. If the user triggers the second interactive control within the preset time period, the multimedia application maintains the display of the second interactive control.

[0208] In one embodiment, the multimedia application may respond to the user's second trigger operation on the second interactive control and update the number and display color of virtual props in the interactive animation in real time according to the number of operations of the second trigger operation.

[0209] In one embodiment, reference Figure 12When a user enters the interactive space, the multimedia application may display a first interactive control 1206. Upon determining that the user has triggered a first trigger action on the first interactive control 1206, the multimedia application may display an interactive animation and a second interactive control 1208. In response to the user's second trigger action on the second interactive control 1208, the multimedia application updates the virtual item storage quantity based on the number of first and second trigger actions. When the virtual item storage quantity reaches zero, the multimedia application transitions to displaying the number of resources required to replenish the virtual item storage quantity. Furthermore, the multimedia application may display a closing countdown 1210 for the interactive space within the multimedia playback interface, thereby reminding the user of the remaining time until the interactive space is closed. While displaying the first and second interactive controls 1206, 1208, the multimedia application may also display the number of all logged-in accounts and the number of first actions 1212 in real time, thereby enhancing the user's sense of shared participation.

[0210] In the above embodiment, by displaying an interactive countdown prompt animation, the user can be prompted with the remaining time to close the interactive space based on the interactive countdown prompt animation, thereby improving the user experience. This application also provides an application scenario, which applies the above virtual prop interaction method. Specifically, the application of the virtual prop interaction method in this application scenario is as follows:

[0211] When a user is watching a TV series through a multimedia application, the multimedia application can display an interactive entrance at a preset position during the TV series playback, so that the user can enter the interactive space by clicking on the interactive entrance. When it is determined that the user has entered the interactive space, the multimedia application can display interactive controls through the multimedia playback interface, and the user can send virtual props to the target object in the TV series by clicking on the interactive controls. When it is determined that the user has clicked on the interactive control, the multimedia application determines the location information of the target object in the TV series, as well as the facial expression of the target object in the TV series, and displays an interactive animation of the virtual props that interacts in an interactive manner that matches the facial expression within the area corresponding to the target object. In this way, users can interact with the target object in the TV series while watching the TV series, greatly enriching the diversity of interactive methods.

[0212] This application also provides another application scenario, which applies the above-mentioned virtual prop interaction method. Specifically, the application of the virtual prop interaction method in this application scenario is as follows:

[0213] When a user is in a video call with a target person through a terminal, the terminal can display an interactive portal and, in response to the user triggering the interactive portal, display interactive controls within the live video playback interface. When the user triggers the interactive control to send a virtual item to the target person, the terminal can determine the target person's target area and facial expression, and display an interactive animation, including a virtual item that interacts with the facial expression, at the location corresponding to the target area within the live video playback interface.

[0214] The above application scenarios are merely illustrative. It is understood that the application of the virtual props interaction method provided in each embodiment of the present application is not limited to the above scenarios.

[0215] In one embodiment, reference Figure 15 ,The virtual prop interaction method includes the following steps:

[0216] S1502: The terminal displays a multimedia playback interface. When it is determined that the multimedia content in the multimedia playback interface has been played to a designated position, an interactive entrance is displayed in the multimedia playback interface.

[0217] S1504: The terminal enters the interactive space in response to the triggering operation on the interactive entrance, and displays the interactive controls of the virtual props in the interactive space.

[0218] S1506, the terminal determines all login accounts currently entering the interactive space, and determines a first operation count for all login accounts to trigger a specified operation on the interactive control together; wherein the login account is an account for viewing multimedia content played on the multimedia playback interface.

[0219] S1508: The terminal determines the number of virtual props displayed in the interactive animation and the display color of the virtual props according to the first operation number.

[0220] S1510, the terminal obtains facial features of the target object, and determines facial key points of the target object based on the facial features, and determines the area where the facial parts of the target object are located and the facial expression of the target object according to the facial key points.

[0221] S1512: When the facial expression of the target object belongs to the first expression category, the terminal determines that the target interaction mode corresponding to the facial expression of the target object is the first interaction mode.

[0222] S1514, when the facial expression of the target object belongs to the second expression category, the terminal determines that the target interaction mode corresponding to the facial expression of the target object is the second interaction mode; wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual props in the first interaction mode is different from that of the virtual props in the second interaction mode.

[0223] S1516, the terminal displays an interactive animation of the virtual props in an area corresponding to the target object in accordance with the display color of the virtual props in the multimedia playback interface, including the number of props and the interactive method matching the facial expression.

[0224] S1518, the terminal determines the number of all login accounts that trigger the specified operation for the interactive control; and displays the number of accounts and the first operation count at a first position corresponding to the interactive control.

[0225] S1520, the terminal determines the storage quantity of virtual props included in the local login account, and displays the storage quantity at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the number of operations of the local login account touching the interactive control accumulates.

[0226] S1522: The terminal displays an interactive countdown prompt animation at a fourth position corresponding to the interactive control, and cancels the display of the interactive control and the interactive animation when the interactive countdown prompt animation ends.

[0227] The above-mentioned virtual props interaction method can display the target object and the interaction entrance through the multimedia playback interface by displaying the multimedia playback interface. By displaying the interaction entrance, in response to the triggering operation of the interaction entrance, an interactive animation including virtual props that interact in an interactive manner that matches the facial expression of the target object is displayed within a range corresponding to the target object. Since an interactive animation including virtual props that interact in an interactive manner that matches the facial expression of the target object can be displayed within a range corresponding to the target object in the multimedia playback interface, compared to the traditional method of participating in the interaction only by sending barrages, the present application can not only establish a strong connection with the target object, but also improve the user experience and increase the diversity of interaction methods with the target object in the multimedia content. In one embodiment, reference Figure 16 , Figure 16The following diagram illustrates the process flow for interacting with virtual props in one embodiment. S1602: When a user clicks an interactive control, S1604: The terminal determines whether the user is logged into the multimedia application. S1606: If the user is not logged in, the terminal displays a login interface, allowing the user to log in through the login interface. S1608: If the user is logged in, the terminal determines whether the logged-in account has any virtual props that can be gifted. S1610: If the logged-in account has any virtual props that can be gifted, the terminal invokes the gifting interface to gift the virtual props to the target object. S1612: The terminal determines whether the gifting interface call is successful. S1614: If the call is successful, the terminal displays an interactive animation. S1616: If the call fails, the terminal displays a prompt indicating that the gifting failed. S1618: If the logged-in account does not have any virtual props that can be gifted, the terminal determines whether the logged-in account has undergone real-name authentication. S1620: If the logged-in account has not undergone real-name authentication, the terminal displays a real-name authentication interface, allowing the user to perform real-name authentication through the interface. S1622: When the login account has been authenticated, the terminal determines whether the login account has sufficient resources to gift virtual items. S1624: When the login account has insufficient resources, the terminal redirects to a resource value replenishment interface, allowing the user to replenish resources through the resource value replenishment interface. When the login account has sufficient resources to gift virtual items, the terminal calls the item gifting interface to gift virtual items to the target object.

[0228] In one embodiment, Figure 17 As shown, a virtual prop interaction method is provided, and the method is described by taking the application of the method to a computer device as an example. The computer device can be Figure 1 The terminal or server in the virtual prop interaction method includes the following steps:

[0229] Step S1702 : In response to a triggering operation on an interactive entry in a multimedia playback interface, a facial expression of a target object in the multimedia content to be played is obtained.

[0230] Specifically, the computer device can load multimedia content to be played. In one embodiment, the computer device can obtain multimedia content in the interactive space and use the multimedia content in the interactive space as the multimedia content to be played. Furthermore, in response to the triggering operation of the interactive entrance in the multimedia playback interface, the computer device determines the facial expression of the target object in each frame of the multimedia content to be played. Among them, the computer device can identify the target object in each frame of the multimedia content to be played through a machine learning model to determine the facial expression of the target object; the computer device can also filter out the target recognition results corresponding to each frame of the multimedia content to be played from a pre-set recognition result library, and determine the facial expression of the target object in each frame of the multimedia content to be played based on the target recognition result.

[0231] In one embodiment, the computer device can also identify a target part of a target object within multimedia content. The target part can be a facial part. When identifying the target object using a machine learning model, the computer device can obtain the target object's facial features and determine the target object's facial key points based on the facial features. The facial features are extracted from a facial image captured of the target object. Based on the facial key points, the computer device determines the target part's location and the target object's facial expression.

[0232] Step S1704 : for the multimedia content to be played, an interactive animation is drawn within the region corresponding to the target object, wherein the interactive animation includes virtual props that interact in a manner that matches the facial expression.

[0233] Specifically, the computer device can determine the area where the target object is located and the target object's facial expression, and determine an interaction method that matches the facial expression. The interaction methods include a first interaction method and a second interaction method. When the target object's facial expression belongs to the first expression category, the interaction method that matches the target object's facial expression is determined to be the first interaction method; when the target object's facial expression belongs to the second expression category, the interaction method that matches the target object's facial expression is determined to be the second interaction method. At least one of the display effect, swing amplitude, and swing frequency of the virtual prop in the first interaction method is different from that of the virtual prop in the second interaction method.

[0234] After determining the target object's area and the interactive method of the virtual prop, the computer device may draw an interactive animation frame within the target object's area within each frame of the multimedia content to be played, and synthesize the interactive animation frames to generate an interactive animation. The interactive animation includes the virtual prop interacting in a manner that matches the facial expression.

[0235] Step S1706: Overlay the multimedia content to be played and the corresponding interactive animation.

[0236] When the interactive animation is drawn, the computer device can superimpose the multimedia content to be played and the corresponding interactive animation to obtain a video for playback. The interactive animation includes at least one interactive animation frame. Among them, the computer device superimposes the multimedia content to be played and the corresponding interactive animation frame at the frame level to obtain the target video for playback. For example, when a virtual prop is drawn within the area where the target object in the multimedia content to be played in frame A is located, and the interactive animation frame a is obtained, the computer device can superimpose the multimedia content to be played in frame A with the interactive animation frame a. In one embodiment, when the server superimposes the multimedia content to be played and the corresponding interactive animation to obtain the target video for playback, the server can send the target video to the terminal so that the terminal plays the target video. In this way, the user can watch the multimedia content and the interactive animation at the same time through the target video played by the terminal.

[0237] In the above-mentioned virtual prop interaction method, by responding to a trigger operation for an interactive entry, multimedia content to be played can be obtained. By obtaining the multimedia content to be played, the facial expression of the target object in the multimedia content to be played can be obtained, and thus an interactive animation can be drawn based on the facial expression. By drawing the interactive animation, the multimedia content to be played and the corresponding interactive animation are superimposed and then played. In this way, an interactive animation including a virtual prop that interacts in an interactive manner that matches the facial expression of the target object can be displayed within the range corresponding to the target object in the multimedia playback interface. This not only establishes a strong connection with the target object, but also improves the user experience and increases the diversity of interaction methods with the target object in the multimedia content. In addition, the device can display interactive animation frames while the user is watching the multimedia content without disturbing the user, enabling the user to obtain a good emotional resonance experience and greatly improving the effectiveness of the interaction. In addition, since the user only needs to trigger the displayed interactive entry to interact with the target object through the virtual prop, the efficiency of human-computer interaction is also greatly improved.

[0238] In one embodiment, obtaining the facial expression of a target object in multimedia content to be played includes: determining the playback progress corresponding to each frame of the multimedia content to be played; filtering target recognition results from a preset recognition result library based on the playback progress corresponding to each frame of the multimedia content to be played; and determining the facial expression of the target object in each frame of the multimedia content to be played based on the target recognition results.

[0239] Specifically, when the computer device determines that the user triggers the interactive entrance, the computer device can determine the playback progress corresponding to each frame of multimedia content to be played, and query the corresponding target recognition results from the recognition result library based on the playback progress, and determine the facial expression of the target object in each frame of multimedia content to be played based on the queried target recognition results.

[0240] In one embodiment, the computer device can obtain the millisecond-level playback progress of the video frame to be played in real time, and based on the millisecond-level playback progress, query the target recognition result corresponding to the video frame to be played from a preset recognition result library, and determine the facial expression of the target object in the video frame to be played based on the target recognition result.

[0241] In one embodiment, when a user triggers an interactive entry, the computer device can simultaneously determine the facial expressions of target subjects within multiple frames of multimedia content to be played. For example, if the computer device needs to play video frames numbered 0001 to 1000 during the target time period from the user entering the interactive space to the user closing the interactive space, the computer device can filter the target recognition results corresponding to each of the frames to be played from a preset recognition result library and, based on the target recognition results, determine the facial expressions of the target subjects within the frames numbered 0001 to 1000.

[0242] In one embodiment, before displaying the interactive entrance, the computer device may pre-acquire the multimedia content to be played by the multimedia application, and identify the target object in each frame of the multimedia content to be played to obtain a recognition result; wherein the recognition result includes the target object area, the target part area and the facial expression of the target object; the recognition results corresponding to each frame of the multimedia content to be played are stored to obtain a recognition result library.

[0243] In the above embodiment, by pre-generating a recognition result library, target recognition results can be quickly screened out from the preset recognition result library, so that the drawing efficiency of the interactive animation frame can be improved based on the quickly screened target recognition results.

[0244] In one embodiment, for multimedia content to be played, a corresponding interactive animation is drawn within an area corresponding to a target object, including: determining candidate display areas for virtual props in each frame of the multimedia content to be played based on a target portion of the target object in each frame of the multimedia content to be played; determining a trajectory curve of the virtual prop based on the candidate display areas, and performing smoothing and optimization processing on the trajectory curve to obtain an optimized trajectory curve; determining a target display area for the virtual prop in each frame of the multimedia content to be played based on the optimized trajectory curve, and drawing the virtual prop within the target display area so as to interact in an interactive manner that matches the facial expression to obtain an interactive animation.

[0245] Specifically, when the target portion of the target object in each frame of the multimedia content to be played is obtained, the computer device can determine the location area where the target portion is located, thereby using the location area where the target portion is located as a candidate display area for the virtual prop. By determining the candidate display areas of the virtual prop in multiple consecutive frames of the multimedia content to be played, the trajectory curve of the virtual prop can be determined based on each candidate display area. To achieve a smooth animation effect, the computer device can use a Kalman filter function (an algorithm that uses a linear system state equation to optimally estimate the system state based on system input and output observation data) to smoothly optimize the trajectory curve to obtain an optimized trajectory curve. Based on the optimized trajectory curve, the target display area of ​​the virtual prop in each frame of the multimedia content to be played is determined, and the virtual prop is drawn within the target display area of ​​the multimedia content to obtain an interactive animation frame. The interactive animation is then synthesized from the interactive animation frames to obtain an interactive animation.

[0246] In this embodiment, by performing smooth optimization processing on the trajectory curve, small-scale glitches in the data can be resolved, and slight jitter in the animation can be avoided, thereby obtaining a smooth interactive animation.

[0247] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0248] In one embodiment, reference Figure 18 , Figure 18The interactive diagram of a virtual prop interactive system in one embodiment is shown. The virtual interactive system includes an application module (also known as a player), a prop interactive module (also known as H5), a data processing module (also known as DataService), and a playback module (also known as PlayerService). When the multimedia content is played to a specified position, the application module can display an interactive entrance, and in response to the user's click operation on the interactive entrance, load the inline frame of the virtual prop (also known as the interactive prop iframe), and send the start and end time of the interactive space to the virtual prop loading module. The prop interactive module can initialize the configuration, initialize the animation rendering component, and initialize the playback module when it is confirmed that the virtual prop is a prop worn by the character object. Furthermore, when it is determined that the user has given the virtual prop to the target object, the data processing module can obtain the current playback progress from the playback module, and filter the target recognition result from the preset recognition result library according to the current playback progress, and return the target recognition result to the prop interactive module. The prop interactive module renders the virtual prop based on the received target recognition result and draws the interactive animation.

[0249] In one embodiment, reference Figure 19 , Figure 19 A schematic diagram of the framework of virtual prop interaction in one embodiment is shown. When the user clicks on the interactive entrance and enters the interactive space, the prop interaction module in the multimedia application can obtain the configuration information of the virtual props, for example, the prop style of the virtual props, the resource value required to be consumed, the interaction method, the power value corresponding to a single virtual prop, and other information. The prop interaction module can also obtain the account information of the login account to determine whether the login account has pre-stored virtual props and whether it has corresponding resource values. When the user clicks on the interactive control to give a virtual prop, the prop interaction module obtains the recognition result and triggers the playback module to load the animation file specified by the virtual prop configuration, initialize the player service, and count down the interactive space. When the interactive space countdown reaches zero, the prop interaction module closes the interactive space and cancels the display of the interactive animation.

[0250] In one embodiment, Figure 20 , provides a logic flow chart for implementing interactive animation. The computer device can identify the position information and facial expression of the target object in the multimedia content through the machine learning model, and perform data smoothing on the position information and facial expression to obtain the smoothed position information and facial expression. Among them, the position information is used to draw the interactive animation, and the facial expression is used to determine the interactive mode of the virtual props in the interactive animation. It is easy to understand, refer to Figure 20 The computer device may also perform data smoothing processing only on the position information of the target object in the multimedia content, and draw an interactive animation based on the position information after the data smoothing processing.

[0251] In one embodiment, reference Figure 21 , Figure 21 A schematic diagram illustrating determining an interaction mode in one embodiment is shown. A computer device can use a machine learning model to identify the facial expression of a target subject in multimedia content. If the facial expression is determined to be stable, the interaction mode is determined to be: default motion, a swing frequency of 2 seconds per round trip, and a swing amplitude of 30 degrees. If the facial expression is determined to be intense, the interaction mode is determined to be: accelerated motion, a swing frequency of 1.2 seconds per round trip, a swing amplitude of 60 degrees, and a particle luminous effect.

[0252] In one embodiment, Figure 22 As shown, a virtual prop interaction device 2200 is provided. The device can be a software module or a hardware module, or a combination of both to form a part of a computer device. The device specifically includes: an interface display module 2202, an entrance display module 2204 and an interaction module 2206, wherein:

[0253] The interface display module 2202 is used to display a multimedia playback interface, in which a target object is displayed.

[0254] The entry display module 2204 is used to display the interactive entry in the multimedia playback interface.

[0255] The interactive module 2206 is used to display an interactive animation in the area corresponding to the target object in the multimedia playback interface in response to the triggering operation of the interactive entrance. The interactive animation includes virtual props that interact in an interactive manner that matches the facial expression of the target object.

[0256] The above-mentioned virtual prop interactive device can display a target object and an interactive portal through a multimedia playback interface. By displaying the interactive portal, in response to a triggering operation on the interactive portal, an interactive animation including a virtual prop that interacts in an interactive manner that matches the target object's facial expression is displayed within a range corresponding to the target object in the multimedia playback interface. Since the interactive animation including the virtual prop that interacts in an interactive manner that matches the target object's facial expression can be displayed within the range corresponding to the target object in the multimedia playback interface, compared to the traditional method of participating in the interaction by sending barrage, this application not only establishes a strong connection with the target object, but also improves the user experience and increases the diversity of interaction methods with the target object in the multimedia content. Moreover, this device can display the interactive portal without disturbing the user's viewing of the multimedia content, allowing the user to conveniently interact with the target object based on the displayed interactive portal. This not only improves the efficiency of interaction, but also enables the user to obtain a good emotional resonance experience, greatly improving the effectiveness of the interaction. In addition, since the user only needs to trigger the displayed interactive portal to interact with the target object through the virtual prop, it also greatly improves the efficiency of human-computer interaction.

[0257] In one embodiment, the interactive module 2206 is also used to detect the area where the target part of the target object played in the multimedia playback interface is located, and the facial expression of the target object; obtain the target interaction method corresponding to the facial expression of the target object; and display an interactive animation including virtual props that interact according to the target interaction method within an area matching the area where the target part is located.

[0258] In one embodiment, the target part includes a facial part, and the interactive module 2206 is further used to obtain facial features of the target object; wherein the facial features are extracted from a facial image collected for the target object; the facial key points of the target object are determined based on the facial features; based on the facial key points, the area where the facial part of the target object is located is determined, and the facial expression of the target object is determined.

[0259] In one embodiment, the target part includes a facial part, and the interactive module 2206 is also used to detect the playback progress of the multimedia content played by the multimedia playback interface; filter out the target recognition results from the preset recognition result library according to the playback progress, and determine the area where the facial part of the target object is located at the current moment according to the target recognition results, and determine the facial expression of the target object.

[0260] In one embodiment, the virtual prop interaction device 2200 is also used to obtain multimedia content played by the multimedia application, and perform frame extraction processing on the multimedia content according to a preset sampling frequency to obtain multiple frames of extracted target video frames; for each frame of the target video frame, the target object in the current target video frame is identified to obtain a recognition result; wherein the recognition result includes the area where the facial parts of the target object are located, and the facial expression of the target object; the recognition results corresponding to each frame of the target video frame are stored to obtain a recognition result library.

[0261] In one embodiment, the virtual prop interaction device 2200 is also used to perform frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain extracted video frames; perform frame missing test on the extracted video frames according to the frame extraction frequency to obtain test results, and based on the test results, perform supplementary frame extraction processing on the multimedia content to obtain supplementary video frames; based on the extracted video frames and the supplementary video frames, obtain multiple frames of target video frames.

[0262] In one embodiment, the virtual prop interaction device 2200 is also used to determine the playback progress corresponding to each target video frame, and divide the playback progress of the target video frame into multiple playback progress intervals according to a preset time window and time window movement step; corresponding to each playback progress interval, determine the difference between the recognition results of each target video frame falling within the current playback progress interval, and merge the recognition results whose differences are less than the difference threshold; store the merged recognition results to obtain a recognition result library.

[0263] In one embodiment, the virtual prop interaction device 2200 is also used to interpolate the recognition results in the recognition result library according to the frame extraction frequency and the frame rate of the multimedia content played by the multimedia playback interface to obtain an interpolation result; based on the interpolation result, determine the correspondence between the position information of the target object in the multimedia content and the playback progress; based on the correspondence, determine the trajectory curve of the virtual prop, and smooth and optimize the trajectory curve to obtain an optimized trajectory curve; display an interactive animation within the area corresponding to the target object in the multimedia playback interface, wherein the virtual props in the interactive animation move according to the optimized trajectory curve.

[0264] In one embodiment, the interaction mode includes the display effect, swing amplitude and swing frequency of the virtual props. The interaction module 2206 is also used to determine that the target interaction mode that matches the facial expression of the target object is the first interaction mode when the facial expression of the target object belongs to the first expression category; when the facial expression of the target object belongs to the second expression category, determine that the target interaction mode that matches the facial expression of the target object is the second interaction mode; wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual props in the first interaction mode is different from that of the virtual props in the second interaction mode.

[0265] In one embodiment, the interaction module 2206 is further configured to respond to a triggering operation on the interaction entrance by displaying an interactive animation in which the virtual prop appears from the interaction entrance, moves toward the facial area of ​​the target object, and interacts.

[0266] In one embodiment, the interaction module 2206 is also used to display an interactive animation. There are multiple virtual props in the interactive animation. After the multiple virtual props move to the area where the facial parts of the target object are located, they surround the facial parts of the target object to interact, and each virtual prop interacts in an interactive manner that matches the facial expression of the target object.

[0267] In one embodiment, the interactive module 2206 is also used to respond to a triggering operation on the interactive entrance, enter the interactive space, and display interactive controls of virtual props in the interactive space; in response to a specified operation on the interactive controls, display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

[0268] In one embodiment, the interactive module 2206 is also used to determine all login accounts currently entering the interactive space, and determine the first operation times for all login accounts to trigger a specified operation for the interactive control; wherein the login account is the account for watching the multimedia content played on the multimedia playback interface; the interactive animation is displayed in the area corresponding to the target object in the multimedia playback interface; wherein the number of virtual props included in the interactive animation is updated in real time as the first operation times accumulate.

[0269] In one embodiment, the interactive module 2206 is further used to determine the number of virtual props displayed in the interactive animation and the display color of the virtual props based on the first operation number; and display the interactive animation within the area corresponding to the target object in the multimedia playback interface based on the display color and the number of virtual props.

[0270] In one embodiment, the virtual object interaction device 2200 is also used to determine the number of all logged-in accounts that jointly trigger a specified operation for the interactive control; and display the number of accounts and the first number of operations at the first position corresponding to the interactive control.

[0271] In one embodiment, the virtual object interaction device 2200 is further used to determine the second operation number of the local login account triggering the specified operation for the interactive control; and display the second operation number at a second position corresponding to the interactive control.

[0272] In one embodiment, the virtual object interaction device 2200 is also used to determine the storage quantity of virtual props included in the local login account; display the storage quantity at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the second operation number accumulates.

[0273] In one embodiment, the virtual object interaction device 2200 is also used to display the resource value required to replenish the storage quantity of the virtual prop when the storage quantity is updated to zero as the second operation number accumulates, and provide a resource value replenishment entry; the resource value replenishment entry is used to implement resource transfer operations.

[0274] In one embodiment, the interactive control includes a first interactive control and a second interactive control. The interactive module 2206 is also used to respond to a trigger operation on the interactive entrance, enter the interactive space, and display the first interactive control in the interactive space; in response to a first trigger operation on the first interactive control, cancel the display of the first interactive control, and switch to displaying the second interactive control, and display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

[0275] In one embodiment, the virtual object interaction device 2200 is also used to display an interactive countdown prompt animation at a fourth position corresponding to the second interactive control in response to a second trigger operation on the second interactive control; when the interactive countdown prompt animation ends, the second interactive control and the interactive animation are canceled.

[0276] In one embodiment, the entry display module 2204 is further configured to display an interactive entry in the multimedia playback interface when it is determined that the multimedia content in the multimedia playback interface has been played to a specified position.

[0277] In one embodiment, Figure 23 As shown, a virtual prop interaction device 2300 is provided. The device can be a software module or a hardware module, or a combination of both to form a part of a computer device. The device specifically includes: an acquisition module 2302, a drawing module 2304 and an overlay module 2306, wherein:

[0278] The acquisition module 2302 is configured to acquire the facial expression of the target object in the multimedia content to be played in response to a triggering operation on the interactive entrance in the multimedia play interface.

[0279] The drawing module 2304 is used to draw the corresponding interactive animation within the area corresponding to the target object for the multimedia content to be played, where the interactive animation includes virtual props that interact in a manner that matches the facial expression.

[0280] The superposition module 2306 is used to superimpose the multimedia content to be played and the corresponding interactive animation.

[0281] In the above-mentioned virtual prop interactive device, by responding to the trigger operation for the interactive entrance, the multimedia content to be played can be obtained. By obtaining the multimedia content to be played, the facial expression of the target object in the multimedia content to be played can be obtained, so that an interactive animation can be drawn based on the facial expression. By drawing the interactive animation, the multimedia content to be played and the corresponding interactive animation can be superimposed and used for playback. In this way, it is possible to display an interactive animation including virtual props that interact in an interactive manner that matches the facial expression of the target object within the range corresponding to the target object in the multimedia playback interface, thereby not only establishing a strong connection with the target object, but also improving the user experience and the diversity of the interaction methods with the target object in the multimedia content. In addition, the device can display interactive animation frames without disturbing the user while watching the multimedia content, which can enable the user to obtain a good emotional resonance experience and greatly improve the effectiveness of the interaction.

[0282] In one embodiment, the acquisition module 2302 is also used to determine the playback progress corresponding to each frame of multimedia content to be played; based on the playback progress corresponding to each frame of multimedia content to be played, the target recognition result is screened out from a preset recognition result library; based on the target recognition result, the facial expression of the target object in each frame of multimedia content to be played is determined.

[0283] In one embodiment, the drawing module 2304 is further configured to determine candidate display areas for the virtual props in each frame of the multimedia content to be played based on the target parts of the target objects in each frame of the multimedia content to be played; determine a trajectory curve for the virtual props based on the candidate display areas, and perform smoothing optimization on the trajectory curve to obtain an optimized trajectory curve; determine a target display area for the virtual props in each frame of the multimedia content to be played based on the optimized trajectory curve, and draw the virtual props within the target display area so as to interact in an interactive manner that matches the facial expression to obtain an interactive animation.

[0284] The specific limitations of the virtual prop interaction device can be found in the limitations of the virtual prop interaction method described above and will not be further elaborated here. Each module in the aforementioned virtual prop interaction device may be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of these modules.

[0285] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 24As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a virtual prop interaction method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0286] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 25 As shown. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store virtual prop interaction data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a virtual prop interaction method is implemented.

[0287] Those skilled in the art will understand that Figure 24-25 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0288] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0289] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0290] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0291] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0292] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0293] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A virtual prop interaction method, characterized in that: The method comprises: Displaying a multimedia playback interface, wherein the multimedia playback interface displays a target object; Displaying an interactive entrance in the multimedia playback interface; In response to a triggering operation on the interactive entrance, an interactive animation is displayed in an area corresponding to the target object in the multimedia playback interface, wherein the interactive animation includes virtual props, and the number of the virtual props is multiple. After the multiple virtual props move to the area where the facial part of the target object is located, they surround the facial part of the target object to interact, and each of the virtual props interacts in an interactive manner that matches the facial expression of the target object.

2. The method according to claim 1, characterized in that The displaying of the interactive animation within the area corresponding to the target object in the multimedia playback interface includes: Detecting an area where a target part of the target object played in the multimedia playback interface is located, and a facial expression of the target object; obtaining a target interaction mode that matches the facial expression of the target object; In an area corresponding to the area where the target part is located, an interactive animation including a virtual prop that interacts according to the target interactive method is displayed.

3. The method according to claim 2, characterized in that The target part includes a facial part, and detecting an area where the target part of the target object played in the multimedia playback interface is located and a facial expression of the target object includes: Acquiring facial features of the target object; wherein the facial features are extracted from a facial image collected from the target object; Determining facial key points of the target object based on the facial features; The facial region of the target object is determined based on the facial key points, and the facial expression of the target object is determined.

4. The method according to claim 2, characterized in that The target part includes a facial part, and detecting an area where the target part of the target object played in the multimedia playback interface is located and a facial expression of the target object includes: Detecting the playback progress of the multimedia content played on the multimedia playback interface; Filter out target recognition results from a preset recognition result library according to the playback progress, and determine the area where the facial part of the target object is located at the current moment and the facial expression of the target object according to the target recognition results.

5. The method according to claim 4, characterized in that The method further comprises: Acquire multimedia content played by a multimedia application, and perform frame extraction on the multimedia content according to a preset frame extraction frequency to obtain multiple extracted target video frames; For each target video frame, the target object in the current target video frame is identified to obtain a recognition result; wherein the recognition result includes the area where the facial part of the target object is located and the facial expression of the target object; The recognition results corresponding to each target video frame are stored to obtain a recognition result library.

6. The method according to claim 5, characterized in that The step of performing frame extraction on the multimedia content according to a preset frame extraction frequency to obtain a plurality of extracted target video frames includes: Performing frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain extracted video frames; Performing a frame missing check on the extracted video frames according to the frame extraction frequency to obtain a check result, and performing a supplementary frame extraction process on the multimedia content according to the check result to obtain a supplementary video frame; Based on the extracted video frames and the supplementary video frames, multiple target video frames are obtained.

7. The method according to claim 5, characterized in that The recognition result corresponding to each target video frame is stored to obtain a recognition result library, including: Determine the playback progress corresponding to each target video frame, and divide the playback progress of the target video frame into multiple playback progress intervals according to a preset time window and a time window movement step; For each playback progress interval, determine the difference between the recognition results of the target video frames falling within the current playback progress interval, and merge the recognition results whose differences are less than the difference threshold; The combined recognition results are stored to obtain a recognition result library.

8. The method according to claim 5, characterized in that The displaying of the interactive animation within the area corresponding to the target object in the multimedia playback interface includes: performing interpolation processing on the recognition results in the recognition result library according to the frame extraction frequency and the frame rate of the multimedia content played by the multimedia playback interface to obtain an interpolation result; determining, based on the interpolation result, a correspondence between the position information of the target object in the multimedia content and the playback progress; Determining a trajectory curve of the virtual prop according to the corresponding relationship, and performing smoothing optimization processing on the trajectory curve to obtain an optimized trajectory curve; An interactive animation is displayed within a region corresponding to the target object in the multimedia playback interface, wherein the virtual props in the interactive animation move according to the optimized trajectory curve.

9. The method according to claim 1, characterized in that The interactive mode includes the display effect, swing amplitude and swing frequency of the virtual prop, and the method further includes: When the facial expression of the target object belongs to the first expression category, determining the target interaction mode that matches the facial expression of the target object as the first interaction mode; When the facial expression of the target object belongs to the second expression category, determining the target interaction mode that matches the facial expression of the target object as the second interaction mode; Wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual prop in the first interactive mode is different from that of the virtual prop in the second interactive mode.

10. The method according to claim 1, characterized in that The step of displaying an interactive animation in a region corresponding to the target object in the multimedia playback interface in response to a triggering operation on the interactive portal comprises: In response to the triggering operation of the interactive entrance, an interactive animation is displayed in which the virtual prop appears from the interactive entrance, moves toward the area where the facial part of the target object is located, and interacts.

11. The method according to claim 1, wherein The step of displaying an interactive animation in a region corresponding to the target object in the multimedia playback interface in response to a triggering operation on the interactive portal comprises: In response to a triggering operation on the interactive entrance, entering the interactive space and displaying interactive controls of virtual props in the interactive space; In response to a designated operation on the interactive control, an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface.

12. The method according to claim 11, characterized in that The step of displaying an interactive animation in a region corresponding to the target object in the multimedia playback interface in response to a designated operation on the interactive control includes: Determining all login accounts currently accessing the interactive space, and determining a first number of operations by all login accounts to trigger a specified operation on the interactive control; wherein all login accounts are accounts that view multimedia content played on the multimedia playback interface; An interactive animation is displayed in an area corresponding to the target object in the multimedia playback interface; wherein the number of virtual props included in the interactive animation is updated in real time as the first operation number accumulates.

13. The method according to claim 12, characterized in that The displaying of the interactive animation within the area corresponding to the target object in the multimedia playback interface includes: determining, according to the first operation count, the number of virtual props displayed in the interactive animation and the display color of the virtual props; An interactive animation is displayed in an area corresponding to the target object in the multimedia playback interface according to the display color of the virtual props and the number of the virtual props.

14. The method according to claim 12, characterized in that After determining the first number of times that all login accounts jointly trigger a specified operation on the interactive control, the method further includes: Determining the number of all login accounts that jointly trigger a specified operation on the interactive control; The number of accounts and the first number of operations are displayed at a first position corresponding to the interactive control.

15. The method according to claim 11, characterized in that The method further comprises: Determine a second operation count of triggering the specified operation on the interactive control by the local login account; The second operation number is displayed at a second position corresponding to the interactive control.

16. The method according to claim 15, characterized in that The method further comprises: Determining the storage quantity of the virtual props included in the local login account; The storage quantity is displayed at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the second operation times accumulates.

17. The method according to claim 16, characterized in that After displaying the stored quantity at a third position corresponding to the interactive control, the method further includes: When the storage quantity is updated to zero as the second operation number accumulates, the resource value required to replenish the storage quantity of the virtual prop is displayed, and a resource value replenishment entry is provided; the resource value replenishment entry is used to implement the resource transfer operation.

18. The method according to claim 11, characterized in that The interactive control includes a first interactive control and a second interactive control. The interactive control for entering the interactive space and displaying the virtual props in the interactive space in response to a triggering operation on the interactive entrance includes: In response to a triggering operation on the interactive entrance, entering the interactive space, and displaying a first interactive control in the interactive space; The step of displaying an interactive animation in a region corresponding to the target object in the multimedia playback interface in response to a designated operation on the interactive control includes: In response to a first triggering operation on the first interactive control, the first interactive control is canceled and the second interactive control is displayed instead, and an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface.

19. The method according to claim 18, characterized in that The method further comprises: In response to a second triggering operation on the second interactive control, displaying an interactive countdown prompt animation at a fourth position corresponding to the second interactive control; When the interactive countdown prompt animation ends, the second interactive control and the interactive animation are canceled.

20. The method according to any one of claims 1 to 19, characterized in that The displaying of the interactive entrance in the multimedia playback interface includes: When it is determined that the multimedia content in the multimedia playback interface is played to a designated position, an interactive entrance is displayed in the multimedia playback interface.

21. A virtual prop interaction method, characterized in that: The method comprises: In response to a triggering operation on an interactive entry in a multimedia playback interface, obtaining a facial expression of a target subject in the multimedia content to be played; For the multimedia content to be played, an interactive animation is drawn within an area corresponding to the target object, wherein the interactive animation includes a virtual prop, and the number of the virtual props is multiple. After the multiple virtual props move to the area where the facial part of the target object is located, they surround the facial part of the target object to interact, and each of the virtual props interacts in an interactive manner that matches the facial expression of the target object; The multimedia content to be played and the corresponding interactive animation are superimposed.

22. A virtual prop interactive device, characterized in that: The device comprises: An interface display module, configured to display a multimedia playback interface, wherein the multimedia playback interface displays a target object; An entry display module, configured to display an interactive entry in the multimedia playback interface; The interactive module is configured to display an interactive animation in an area corresponding to the target object in the multimedia playback interface in response to a triggering operation on the interactive portal, wherein the interactive animation includes a plurality of virtual props, and after the plurality of virtual props move to the area where the facial part of the target object is located, they surround the facial part of the target object to interact, and each of the virtual props interacts in an interactive manner that matches the facial expression of the target object.

23. The device according to claim 22, characterized in that The interactive module is also used to detect the area where the target part of the target object played in the multimedia playback interface is located, and the facial expression of the target object to obtain a target interaction method that matches the facial expression of the target object; within the area corresponding to the area where the target part is located, an interactive animation including virtual props that interact according to the target interaction method is displayed.

24. The device according to claim 23, characterized in that The target parts include facial parts, and the interactive module is further used to obtain facial features of the target object; wherein the facial features are extracted from a facial image collected for the target object; the facial key points of the target object are determined based on the facial features; based on the facial key points, the area where the facial parts of the target object are located is determined, and the facial expression of the target object is determined.

25. The device according to claim 23, characterized in that The target part includes a facial part, and the interactive module is further used to detect the playing progress of the multimedia content played on the multimedia playing interface; Filter out target recognition results from a preset recognition result library according to the playback progress, and determine the area where the facial part of the target object is located at the current moment and the facial expression of the target object according to the target recognition results.

26. The device according to claim 25, characterized in that The virtual prop interaction device is further used to obtain multimedia content played by the multimedia application, and perform frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain multiple extracted target video frames; For each target video frame, the target object in the current target video frame is identified to obtain a recognition result; wherein the recognition result includes the area where the facial parts of the target object are located and the facial expression of the target object; the recognition results corresponding to each target video frame are stored to obtain a recognition result library.

27. The device according to claim 26, characterized in that The virtual prop interaction device is further configured to perform frame extraction processing on the multimedia content according to a preset frame extraction frequency to obtain extracted video frames; The extracted video frames are subjected to a frame missing check according to the frame extraction frequency to obtain a check result, and based on the check result, the multimedia content is subjected to a supplementary frame extraction process to obtain a supplementary video frame; based on the extracted video frames and the supplementary video frames, a plurality of target video frames are obtained.

28. The device according to claim 26, characterized in that The virtual prop interaction device is also used to determine the playback progress corresponding to each target video frame, and divide the playback progress of the target video frame into multiple playback progress intervals according to a preset time window and time window movement step; corresponding to each playback progress interval, determine the difference between the recognition results of each target video frame falling within the current playback progress interval, and merge the recognition results whose differences are less than the difference threshold; store the merged recognition results to obtain a recognition result library.

29. The device according to claim 26, characterized in that The virtual prop interaction device is further used to interpolate the recognition results in the recognition result library according to the frame extraction frequency and the frame rate of the multimedia content played by the multimedia playback interface to obtain an interpolation result; determine the correspondence between the position information of the target object in the multimedia content and the playback progress according to the interpolation result; determine the trajectory curve of the virtual prop according to the correspondence, and smooth and optimize the trajectory curve to obtain an optimized trajectory curve; display an interactive animation within the area corresponding to the target object in the multimedia playback interface, wherein the virtual prop in the interactive animation moves according to the optimized trajectory curve.

30. The device according to claim 22, wherein The interaction mode includes the display effect, swing amplitude and swing frequency of the virtual prop. The interaction module is further used to determine that the target interaction mode that matches the facial expression of the target object is the first interaction mode when the facial expression of the target object belongs to the first expression category; when the facial expression of the target object belongs to the second expression category, determine that the target interaction mode that matches the facial expression of the target object is the second interaction mode; wherein, at least one of the display effect, swing amplitude and swing frequency of the virtual prop in the first interaction mode is different from that of the virtual prop in the second interaction mode.

31. The device according to claim 22, characterized in that The interactive module is further configured to respond to a triggering operation on the interactive entrance by displaying an interactive animation in which a virtual prop appears from the interactive entrance, moves toward the area where the facial part of the target object is located, and interacts.

32. The device according to claim 22, characterized in that The interactive module is also used to respond to a triggering operation on the interactive entrance, enter the interactive space, and display interactive controls of virtual props in the interactive space; in response to a specified operation on the interactive control, display an interactive animation within the area corresponding to the target object in the multimedia playback interface.

33. The device according to claim 32, characterized in that The interactive module is also used to determine all login accounts currently entering the interactive space, and determine the first number of operations for all login accounts to trigger a specified operation for the interactive control; wherein, all login accounts are accounts that watch the multimedia content played on the multimedia playback interface; and display an interactive animation within the area corresponding to the target object in the multimedia playback interface; wherein, the number of virtual props included in the interactive animation is updated in real time as the first number of operations accumulates.

34. The device according to claim 33, characterized in that The interactive module is also used to determine the number of virtual props displayed in the interactive animation and the display color of the virtual props based on the first operation number; and display the interactive animation within the area corresponding to the target object in the multimedia playback interface based on the display color of the virtual props and the number of virtual props.

35. The device according to claim 33, characterized in that The virtual props interaction device is also used to determine the number of all logged-in accounts that jointly trigger a specified operation for the interactive control; and display the number of accounts and the first operation times at a first position corresponding to the interactive control.

36. The device according to claim 32, characterized in that The virtual prop interaction device is further used to determine a second operation number of the local login account triggering the specified operation for the interactive control; and display the second operation number at a second position corresponding to the interactive control.

37. The device according to claim 36, characterized in that The virtual props interaction device is also used to determine the storage quantity of the virtual props included in the local login account; display the storage quantity at a third position corresponding to the interactive control; wherein the storage quantity is updated in real time as the second operation number accumulates.

38. The device according to claim 37, characterized in that The virtual prop interactive device is also used to display the resource value required to replenish the storage quantity of the virtual prop when the storage quantity is updated to zero as the second operation number accumulates, and provide a resource value replenishment entry; the resource value replenishment entry is used to implement the resource transfer operation.

39. The device according to claim 32, characterized in that The interactive control includes a first interactive control and a second interactive control, and the interactive module is further configured to enter the interactive space in response to a triggering operation on the interactive entrance, and display the first interactive control in the interactive space; In response to a first triggering operation on the first interactive control, the first interactive control is canceled and the second interactive control is displayed instead, and an interactive animation is displayed within an area corresponding to the target object in the multimedia playback interface.

40. The device according to claim 39, characterized in that The virtual prop interaction device is also used to display an interactive countdown prompt animation at a fourth position corresponding to the second interactive control in response to a second trigger operation on the second interactive control; when the interactive countdown prompt animation ends, the second interactive control and the interactive animation are canceled.

41. The device according to claim 22, characterized in that The entry display module is further configured to display an interactive entry in the multimedia playback interface when it is determined that the multimedia content in the multimedia playback interface has been played to a designated position.

42. A virtual prop interactive device, characterized in that: The device comprises: An acquisition module, configured to acquire the facial expression of a target object in the multimedia content to be played in response to a triggering operation on an interactive entry in the multimedia playback interface; a drawing module configured to draw, for the multimedia content to be played, an interactive animation corresponding to the target object within an area corresponding to the target object, wherein the interactive animation includes a plurality of virtual props, wherein the plurality of virtual props move to an area where the target object's face is located and then surround the target object's face to interact, with each virtual prop interacting in a manner that matches the target object's facial expression; The superposition module is used to superimpose the multimedia content to be played and the corresponding interactive animation.

43. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 21 are implemented.

44. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

45. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

Images