Powerpoint operation method and device, electronic equipment, storage medium and computer program product

Abstract

The embodiment of the invention discloses a PowerPoint operation method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps of obtaining an object image of a target object, and determining a presentation file operation of the target object for a displayed target presentation file according to the object image; and displaying the object image, and synchronously operating the target PowerPoint according to the PowerPoint operation. According to the technical scheme provided by the embodiment of the invention, the synchronization of displaying the object image and operating the presentation file is realized.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular, to a presentation operation method, device, electronic device, storage medium and computer program product. Background Art

[0002] Operating on the PowerPoint Presentation (PPT) is a very important part in the PPT presentation process.

[0003] However, the current way of operating PPT will cause the displayed object image of the target object of the PPT operation to be out of sync with the operation of the PPT, which needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present invention provide a presentation operation method, device, electronic device, storage medium and computer program product, which realize the synchronization of displaying an object image and operating a presentation.

[0005] According to one aspect of the present invention, a presentation operation method is provided, which may include:

[0006] Acquire an object image of a target object, and determine a presentation operation of the target object with respect to a displayed target presentation according to the object image;

[0007] Display the object image and synchronously operate the target presentation according to the presentation operation.

[0008] According to another aspect of the present invention, a presentation operation device is provided, which may include:

[0009] A presentation operation determination module, used to obtain an object image of a target object, and determine a presentation operation of the target object for a displayed target presentation according to the object image;

[0010] The target presentation operation module is used to display the object image and synchronously operate the target presentation according to the presentation operation.

[0011] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0012] at least one processor; and

[0013] a memory communicatively connected to at least one processor; wherein,

[0014] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the presentation operation method provided by any embodiment of the present invention when executing the computer program.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the presentation operation method provided by any embodiment of the present invention when executed.

[0016] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the presentation operation method provided by any embodiment of the present invention.

[0017] The technical solution of the embodiment of the present invention obtains the object image of the target object, and determines the presentation operation of the target object for the displayed target presentation according to the object image, so as to determine the operation required for the displayed target presentation; displays the object image, and synchronously operates the target presentation according to the presentation operation, so as to display the object image and operate the presentation. The above technical solution, after obtaining the object image, first determines the presentation operation, then displays the object image, and synchronously operates the target presentation according to the presentation operation, thereby avoiding the problem of the object image of the target object for operating the presentation being out of sync with the operation on the presentation due to the independence of displaying the object image and determining the presentation operation, and realizes the synchronization of displaying the object image and operating the presentation.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a presentation operation method provided according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of another presentation operation method provided according to an embodiment of the present invention;

[0022] Figure 3 is a flow chart of another presentation operation method provided according to an embodiment of the present invention;

[0023] Figure 4 is a structural diagram of an optional example of another presentation operation method provided according to an embodiment of the present invention;

[0024] Figure 5 is a structural block diagram of a presentation operation device provided according to an embodiment of the present invention;

[0025] Figure 6 It is a structural schematic diagram of an electronic device for implementing the presentation operation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Before introducing the embodiments of the present invention, an exemplary explanation is given of the implementation process of the current presentation operation scheme and the reasons for the problem that the object image of the target object of the PPT operation is not synchronized with the operation of the PPT, so as to better understand why the scheme proposed in the embodiments of the present invention achieves the synchronization of displaying the object image and operating the presentation.

[0029] For example, traditional PPT operation methods such as mouse clicks or keyboard operations are not convenient enough in some specific occasions such as remote demonstrations and teaching. At present, with the continuous development of computer vision and artificial intelligence technology, the way of operating PPT presentations based on the object image of the target object in the process of demonstration or teaching has gradually attracted attention. However, since the live broadcast and other video streams composed of the object image of the target object in the process of demonstration or teaching will be displayed independently, and the time consumed by the process of determining the presentation operation using computer vision and artificial intelligence technology will result in the mutual independence of displaying the object image and determining the presentation operation, so that the time for operating the target presentation is lagging compared to the time when there is an object image indicating the operation of the target presentation, which leads to the problem that the displayed object image of the target object operating the PPT is not synchronized with the operation of the PPT.

[0030] In view of this, the embodiment of the present invention determines the presentation operation first after acquiring the object image, then displays the object image, and synchronously operates the target presentation according to the presentation operation, thereby avoiding the problem that the object image of the target object of the presentation operation is out of sync with the operation on the presentation due to the independence of displaying the object image and determining the presentation operation, thereby achieving synchronization of displaying the object image and operating the presentation. This will be explained in detail below.

[0031] Figure 1 1 is a flowchart of a presentation operation method provided in an embodiment of the present invention. This embodiment is applicable to the case of presentation operation. The method can be executed by a presentation operation device provided in an embodiment of the present invention, the device can be implemented by software and / or hardware, the device can be integrated in an electronic device, and the electronic device can be various user terminals or servers.

[0032] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0033] S110, acquiring an object image of the target object, and determining, based on the object image, a presentation operation of the target object with respect to the displayed target presentation.

[0034] The target object is an object that may require a presentation operation; the target object may be an object being broadcast live, demonstrated, or taught. The object image is an image obtained by image acquisition of the target object. The target presentation is the presentation that requires an operation; the target presentation may be, for example, a presentation displayed on the target object end corresponding to the target object, or may be, for example, a presentation displayed on the viewing object end that views the object image, and so on. The presentation operation is an operation on the target presentation, and the presentation operation may be, for example, page turning, closing, presentation projection, or exit projection, and so on.

[0035] In the embodiment of the present invention, for example, a camera for video acquisition of a target object is connected to obtain a live video of the target object in real-time video broadcast, and a video stream is obtained by decoding the live video, and a video frame in the video stream is used as an object image, so as to achieve synchronization between live broadcast and operation presentation in scenarios such as live broadcast where synchronization requirements for displaying object images and operating presentations are high. In the embodiment of the present invention, the method for obtaining the object image of the target object is not specifically limited.

[0036] In the embodiment of the present invention, for example, the operation gesture of the target object can be determined based on the object image, and the presentation operation of the target object on the displayed target presentation can be determined based on the operation gesture. In the embodiment of the present invention, the presentation operation method of the displayed target presentation is not specifically limited.

[0037] In an embodiment of the present invention, before determining the target object for the presentation operation of the displayed target presentation based on the object image, object recognition may be performed on the object image, an object area corresponding to the target object may be determined from the object image, and the object image may be updated based on the object area.

[0038] S120, display the object image, and synchronously operate the target presentation according to the presentation operation.

[0039] In the embodiment of the present invention, the object image can be displayed, and the operation can be performed according to the presentation at the same time, and the target presentation can be operated contactlessly, which can not only facilitate the operation of the presentation in some specific occasions such as remote presentation and teaching, but also realize the synchronization of displaying the object image and operating the presentation. In the embodiment of the present invention, the method of displaying the object image, and operating according to the presentation at the same time, and operating the target presentation is not specifically limited.

[0040] The technical solution of the embodiment of the present invention obtains the object image of the target object, and determines the presentation operation of the target object for the displayed target presentation according to the object image, so as to determine the operation required for the displayed target presentation; displays the object image, and synchronously operates the target presentation according to the presentation operation, so as to display the object image and operate the presentation. The above technical solution, after obtaining the object image, first determines the presentation operation, then displays the object image, and synchronously operates the target presentation according to the presentation operation, thereby avoiding the problem of the object image of the target object for operating the presentation being out of sync with the operation on the presentation due to the independence of displaying the object image and determining the presentation operation, and realizes the synchronization of displaying the object image and operating the presentation.

[0041] An optional technical solution, a presentation operation method, also includes: acquiring voice data of a target object, and parsing the voice data to obtain parsed data; determining the presentation operation of the target object for a displayed target presentation based on an object image, including: when the parsed data indicates that the target object has an operation action, determining the presentation operation of the target object for the displayed target presentation based on the object image.

[0042] Among them, language data is data obtained by voice collection of the target object; the voice of the target object for a preset collection time can be periodically or continuously collected as voice data. Analysis data is data obtained by analyzing voice data; analysis data can be obtained, for example, by converting voice data into text, etc. In the embodiment of the present invention, the voice data is analyzed and the method of obtaining the analysis data is not specifically limited. An operation action is something that can operate the target presentation, that is, for example, the action of raising a hand can turn the page of the target presentation, so the action of raising a hand is an operation action, and the action of waving a hand is not set to operate the target presentation, so the action of waving a hand is not an operation action.

[0043] Exemplarily, voice data of the target object can be acquired and analyzed to obtain analysis data in text form or the like. When the analysis data includes preset prompt words such as "watch my actions", "I want to operate" or "turn pages" that can determine that the analysis data indicates that the target object has an operation action, the presentation operation of the target object for the displayed target presentation can be determined based on the object image.

[0044] In the embodiment of the present invention, for example, when the parsed data indicates that the target object has an operation action, the action data of the target object can be determined based on the object image, and the presentation operation of the target object for the displayed target presentation can be determined based on the action data. In the embodiment of the present invention, there is no specific limitation on the method of determining the presentation operation of the target object for the displayed target presentation based on the object image when the parsed data indicates that the target object has an operation action.

[0045] In an embodiment of the present invention, when the analyzed data indicates that the target object has an operation action, an object image of the target object can be obtained, or an object image collected within a preset time period corresponding to the voice acquisition time can be obtained, so as to better perform scenario-based presentation operations. The voice acquisition time is the time when the voice data is acquired or the time corresponding to the voice point in the voice data corresponding to the prompt word; the preset time period is the time period for collecting the object image, and the preset time period can be, for example, a time period of the first preset time length after the voice acquisition time and / or a time period of the second preset time length after the voice acquisition time.

[0046] In the embodiment of the present invention, when the parsed data indicates that there is no operation action on the target object, the object image may be displayed.

[0047] In an embodiment of the present invention, by analyzing the voice data and obtaining analysis data representing the existence of an operation action of the target object, the presentation operation of the target object with respect to the displayed target presentation is determined according to the object image, thereby avoiding misjudgment of the presentation operation and improving the accuracy of the operation on the target presentation.

[0048] Based on the above scheme, another optional technical scheme is to determine the presentation operation of the target object for the displayed target presentation according to the object image, including: determining the presentation operation of the target object for the displayed target presentation according to the object image and the parsed data.

[0049] In the embodiment of the present invention, since the parsed data may include prompt words such as "raise your hand" or "turn the page", the presentation operation of the target object for the displayed target presentation can be determined based on the object image and the parsed data, thereby improving the accuracy of the determined presentation operation. For example, the object image and the parsed data can be input into the action recognition model to obtain the action data, and the presentation operation of the target object for the displayed target presentation can be determined based on the action data. In the embodiment of the present invention, the method of determining the presentation operation of the target object for the displayed target presentation based on the object image and the parsed data is not specifically limited.

[0050] In an embodiment of the present invention, by jointly determining the presentation operation of the target object for the displayed target presentation based on the object image and the parsed data, the accuracy of the determined presentation operation can be improved, thereby improving the accuracy of the operation on the target presentation.

[0051] Figure 2 It is a flow chart of another presentation operation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, according to the object image, the presentation operation of the target object for the displayed target presentation is determined, including: according to the object image, determining the action data of the target object; according to the action data, determining the presentation operation of the target object for the displayed target presentation.

[0052] The explanations of the terms that are the same as or corresponding to the above embodiments are not repeated here.

[0053] See also Figure 2 The method of this embodiment may specifically include the following steps:

[0054] S210: Acquire an object image of the target object.

[0055] S220: Determine the action data of the target object according to the object image.

[0056] Among them, the action data is data related to the action of the target object represented by the object image; the action data can be, for example, actions such as "raising hand" or "waving hand", or can be extracted mark points or action contours of the target object, etc. In the embodiment of the present invention, there is no specific limitation on the form of expression of the action data.

[0057] In the embodiment of the present invention, the action data of the target object may be determined according to the object image by, for example, a model or image recognition method. In the embodiment of the present invention, the method of determining the action data of the target object according to the object image is not specifically limited.

[0058] S230. Determine, based on the action data, the presentation operation of the target object with respect to the displayed target presentation.

[0059] In an embodiment of the present invention, for example, an alternative operation having a mapping relationship with the action data can be determined from at least one alternative operation as a presentation operation of the target object for the displayed target presentation. If an alternative operation having a mapping relationship with the action data can no longer be determined from at least one alternative operation, it means that the target object currently does not need to operate the target presentation, and only needs to display the object image. In an embodiment of the present invention, there is no specific limitation on the method of determining the presentation operation of the target object for the displayed target presentation based on the action data.

[0060] The technical solution of the embodiment of the present invention can realize the operation of the target presentation through the action of the target object, by determining the action data of the target object according to the object image, and determining the presentation operation of the target object with respect to the displayed target presentation according to the action data. The target presentation does not need to be operated by the target object through a mouse or keyboard, thereby realizing more convenient and quick operation on the target presentation.

[0061] An optional technical solution determines the presentation operation of the target object for a displayed target presentation based on action data, including: when the action data meets preset operation conditions, determining the presentation operation of the target object for the displayed target presentation based on the action data.

[0062] The preset operation conditions are conditions that must be met before the presentation operation can be determined.

[0063] In an embodiment of the present invention, considering that some actions of the target object are actions for indicating the operation of a presentation, while some actions are not actions for indicating the operation of a presentation, therefore, the preset operation condition may, for example, be the ability to determine the operation gesture of the target object based on the action data, that is, only when the operation gesture of the target object can be determined based on the action data, the presentation operation of the target object with respect to the displayed target presentation will be determined based on the action data; considering that in scenarios such as live broadcast, an action of the target object indicating the operation of a presentation may correspond to multiple object images, in order to avoid the problem of multiple operations of the presentation with one action indicating the operation of a presentation, the preset operation condition may, for example, be that the object image is an operation image in a set of action images, that is, only when the object image is an operation image, the presentation operation of the target object with respect to the displayed target presentation will be determined based on the action data; the preset operation condition may, for example, be that the object image is an operation image in a set of action images and the operation gesture of the target object can be determined based on the action data. Among them, the action image set is the image set corresponding to the current action group where the action represented by the object image is located; the current action group is a complete action combination of the action represented by the object image, for example, the current action group is a complete action combination constituting the action of raising the hand, and the action represented by the object image is the action of raising the hand above the head in the complete action combination of the action of raising the hand; the operation image is an image that can reflect the action of the current action group, for example, the first image, the last image, the middle image, the preset number of images or the image that best reflects the action of the current action group in the action image set corresponding to the current action group, etc. In the embodiment of the present invention, the content of the preset operation condition is not specifically limited.

[0064] In an embodiment of the present invention, when the action data satisfies a preset operation condition, for example, the operation gesture of the target object can be determined based on the action data, and an alternative operation having a mapping relationship with the operation gesture is determined from at least one alternative operation as the presentation operation of the target object for the displayed target presentation. In an embodiment of the present invention, there is no specific limitation on the manner of determining the presentation operation of the target object for the displayed target presentation based on the action data when the action data satisfies the preset operation condition.

[0065] In an embodiment of the present invention, when the action data meets the preset operation conditions, the presentation operation of the target object for the target presentation being displayed is determined based on the action data, which can avoid the situation where an erroneous presentation operation is determined when the action data does not meet the preset operation conditions, thereby causing an erroneous operation on the target presentation.

[0066] Based on the above solution, another optional technical solution, the presentation operation method, also includes: displaying the object image when the action data does not meet the preset operation conditions.

[0067] In the embodiment of the present invention, when the action data does not meet the preset operation conditions, it means that there is no need to operate the presentation, and there is no need to display the object image and synchronize the operation presentation, and the object image can be directly displayed. The above technical solution can continue to display the object image without operating the presentation.

[0068] Another optional technical solution is to determine the motion data of the target object based on the object image, including: obtaining a motion recognition model; and determining the motion data of the target object based on the object image and the motion recognition model.

[0069] Among them, the action recognition model is a model that can recognize the action of a captured target object based on the object image.

[0070] It should be noted that currently, wearable devices or markers are usually used to track actions to determine action data, but the way the action data is determined limits its application flexibility in actual scenarios, and there are problems such as high cost, complex configuration environment, inability to combine with currently used scenarios, and inability to better reflect action intentions. Therefore, in an embodiment of the present invention, an action recognition model such as a large skeletal action model can be used to capture the skeletal actions of the target object in the object image, identify common gesture actions and / or body actions, and thereby determine the action data of the target object to avoid the occurrence of the above-mentioned problems.

[0071] In the embodiment of the present invention, for example, the key points on the hand and / or body of the target object in the object image can be extracted through the action recognition model, and the action data of the target object can be determined based on the key points. In the embodiment of the present invention, the method of determining the action data of the target object based on the object image and the action recognition model is not specifically limited.

[0072] In an embodiment of the present invention, the action recognition model is a model that can perform action recognition on an object image to obtain the action data of the target object. The action recognition model can be, for example, a suitable large skeleton action model or a one-time detection (You Only Look Once, YOLO) model, etc., and in an embodiment of the present invention, the type of the action recognition model is not specifically limited. Before obtaining the action recognition model, a sample image including gestures and / or body movements can be obtained, for example, a sample image including gestures and / or body movements related to operating a presentation can be obtained, and a large skeleton action model or a YOLO model for action recognition is used according to a data set composed of a plurality of sample images. Models such as the large skeleton action model or the YOLO model are trained or fine-tuned using an enlarged model unsupervised learning and / or fine-tuning supervised learning technology, so that the action recognition model obtained by training or fine-tuning can determine more accurate and more targeted action data for gestures and / or body movements according to the object image without marking.

[0073] In an embodiment of the present invention, different target objects may have different action habits for operating a presentation, for example, target object A is accustomed to raising his hand to turn pages, while target object B is accustomed to waving his hand to turn pages. In order to enable the action recognition model to determine more targeted and accurate action data for gesture actions and / or body actions related to operating the presentation based on the object image, the input action can be obtained, that is, the action that the input target object is accustomed to operating the presentation, and the action recognition model can be adjusted according to the input action. For example, parameters related to weights in the action recognition model can be adjusted, or sample data related to the input action can be obtained to train the action recognition model, and the action recognition model can be updated according to the adjustment result.

[0074] In the embodiment of the present invention, by determining the motion data of the target object according to the object image and the motion recognition model, the accuracy of the determined motion data can be improved.

[0075] Figure 3It is a flowchart of another presentation operation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, according to the action data, the presentation operation of the target object for the displayed target presentation is determined, including: determining the operation gesture of the target object according to the action data; and determining, from at least one alternative operation, an alternative operation having a mapping relationship with the operation gesture as the presentation operation of the target object for the displayed target presentation.

[0076] The explanations of the terms that are the same as or corresponding to the above embodiments are not repeated here.

[0077] See also Figure 3 The method of this embodiment may specifically include the following steps:

[0078] S310: Acquire an object image of a target object.

[0079] S320: Determine the action data of the target object according to the object image.

[0080] S330: Determine the operation gesture of the target object according to the action data.

[0081] Among them, the operation gesture is the gesture that the target object needs to operate the target presentation.

[0082] In the embodiment of the present invention, for example, when the action data is an action such as "raising hand" or "waving hand", the action data can be directly used as an operation gesture, or when there is an alternative gesture corresponding to the action data in at least one alternative gesture, the action data can be used as an operation gesture, and when there is no alternative gesture corresponding to the action data in at least one alternative gesture, the object image can be directly displayed, wherein the alternative gesture is an alternative gesture that can operate the target presentation; for another example, when the action data is data related to the action such as the extracted mark point or action outline of the target object, the action gesture can be determined according to the action data, and the action gesture can be directly used as an operation gesture, or when there is an alternative gesture corresponding to the action gesture in at least one alternative gesture, the action gesture can be used as an operation gesture, and when there is no alternative gesture corresponding to the action gesture in at least one alternative gesture, the object image can be directly displayed; and so on. In the embodiment of the present invention, there is no specific limitation on the method of determining the operation gesture of the target object according to the action data.

[0083] S340. Determine, from at least one candidate operation, an alternative operation that has a mapping relationship with the operation gesture as a presentation operation of the target object for the displayed target presentation.

[0084] Among them, the alternative operation is an alternative operation that can operate the target presentation. The mapping relationship can represent the corresponding relationship between at least one alternative operation and each operation gesture; in an embodiment of the present invention, the mapping relationship between at least one alternative operation and each operation gesture can be predefined, for example, the operation gesture of waving right is mapped to the alternative operation of "next page".

[0085] In an embodiment of the present invention, at least one alternative operation may be, for example, various simulated keyboard events, keyboard input operations or key functions related to the operation of the presentation, such as "previous page" or "next page", so that the presentation operation may be a simulated keyboard event, keyboard input operation or key function, thereby realizing the operation of the target presentation.

[0086] In the embodiment of the present invention, there is no specific limitation on the manner in which an alternative operation having a mapping relationship with the operation gesture is determined from at least one alternative operation as the presentation operation of the target object for the displayed target presentation.

[0087] In an embodiment of the present invention, due to different target objects, the action habits for operating the presentation may be different. For example, target object C is accustomed to raising his hand to turn pages, while target object D is accustomed to waving his hand to turn pages. In order to determine a more accurate presentation operation by using the operation gesture corresponding to the action that satisfies the target object's habits, the input action and the input operation corresponding to the input action can be obtained, at least one alternative operation can be updated according to the input operation, and the mapping relationship can be updated according to the input action and the input operation.

[0088] S350, display the object image, and synchronously operate the target presentation according to the presentation operation.

[0089] In an embodiment of the present invention, real-time and accuracy can be considered in the design of the presentation operation method, and algorithm optimization can be performed to ensure that the operation gestures can be accurately recognized and the target presentation can be operated with low latency.

[0090] The technical solution of the embodiment of the present invention determines the operation gesture of the target object according to the action data, and determines the alternative operation having a mapping relationship with the operation gesture from at least one alternative operation, as the presentation operation of the target object for the displayed target presentation. The presentation operation can be quickly and accurately determined, thereby improving the display object image, and synchronously operating the target presentation according to the presentation operation, and improving the accuracy of operating the target presentation.

[0091] An optional technical solution, a presentation operation method, also includes: determining the body movement of the target object based on the action data; determining an alternative operation having a mapping relationship with the operation gesture from at least one alternative operation, as the presentation operation of the target object for the displayed target presentation, including: determining an alternative operation having a mapping relationship with the operation gesture and the body movement from at least one alternative operation, as the presentation operation of the target object for the displayed target presentation.

[0092] Among them, the body action is the body action that the target object needs to operate the target presentation.

[0093] In the embodiment of the present invention, for example, when the action data is an action such as "nodding", the action data can be directly used as a body action, or when there is an alternative action corresponding to the action data in at least one alternative action, the action data can be used as a body action, wherein the alternative action is an alternative action that can operate the target presentation; for another example, when the action data is data related to the action such as the extracted mark points or action contours of the target object, the body data can be determined according to the action data, and the body data can be directly used as an operation gesture, or when there is an alternative action corresponding to the body data in at least one alternative action, the body data can be used as a body action; and so on. In the embodiment of the present invention, there is no specific limitation on the method of determining the body action of the target object according to the action data.

[0094] In the embodiment of the present invention, there is no specific limitation on the manner of determining, from at least one candidate operation, an alternative operation having a mapping relationship with an operation gesture and a body movement as a presentation operation of a target object for a displayed target presentation.

[0095] In an embodiment of the present invention, an alternative operation having a mapping relationship with a body movement can also be determined from at least one alternative operation as a presentation operation of the target object for the displayed target presentation; an alternative operation having a mapping relationship with at least one of an operation gesture and a body movement can also be determined from at least one alternative operation as a presentation operation of the target object for the displayed target presentation.

[0096] In an embodiment of the present invention, by determining, from at least one alternative operation, an alternative operation having a mapping relationship with an operation gesture and a body movement as a presentation operation of a target object for a displayed target presentation, the accuracy of the determined presentation operation can be improved.

[0097] In order to better understand the technical solution of the above embodiment of the present invention, an optional example is provided here. Figure 4The presentation operation method can be applied to a presentation operating system; the presentation operating system may include at least one camera, a skeleton action model and a processing module, and the processing module may include a live broadcast module, a gesture recognition module and a keyboard simulation module; at least one camera can perform video acquisition on a target image to obtain a video stream, and use the video frames in the video stream as the object image; the processing module can obtain the object image and voice data in the form of frames, determine the prompt word according to the voice data, and send the object image and the prompt word to the skeleton action model, so that the skeleton action model determines the action data according to the object image and the prompt word, and sends the action data to the processing module; the gesture recognition module in the processing module can determine the operation gesture according to the action data; the keyboard simulation module in the processing module can determine the keyboard event as the presentation operation according to the operation gesture; the live broadcast module in the processing module can broadcast and display the object image, and the processing module can synchronously perform operations such as turning pages on the target presentation according to the presentation operation through the presentation control module and other parts therein.

[0098] Figure 5 This is a structural block diagram of a presentation operation device provided in an embodiment of the present invention, and the device is used to execute the presentation operation method provided in any of the above embodiments. The device and the presentation operation method in the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the presentation operation device can be referred to the embodiment of the above presentation operation method. Figure 5 , the device may specifically include: a presentation operation determination module 410 and a target presentation operation module 420.

[0099] The presentation operation determination module 410 is used to obtain an object image of a target object, and determine the presentation operation of the target object for the displayed target presentation according to the object image;

[0100] The target presentation operation module 420 is used to display the object image and synchronously operate the target presentation according to the presentation operation.

[0101] Optionally, the presentation operation determination module 410 includes:

[0102] An action data determination submodule, used to determine the action data of the target object according to the object image;

[0103] The first presentation operation determination submodule is used to determine the presentation operation of the target object for the displayed target presentation according to the action data.

[0104] Optionally, based on the above device, the first presentation operation determination submodule includes:

[0105] An operation gesture determining unit, used to determine the operation gesture of the target object according to the action data;

[0106] The presentation operation is used as a unit to determine, from at least one candidate operation, an alternative operation having a mapping relationship with the operation gesture as a presentation operation of the target object for the displayed target presentation.

[0107] Optionally, based on the above device, the device further includes:

[0108] A body motion determination module, used to determine the body motion of the target object based on the motion data;

[0109] Presentation operations as a unit include:

[0110] The presentation operation is used as a subunit to determine, from at least one candidate operation, an alternative operation having a mapping relationship with the operation gesture and the body movement as a presentation operation of the target object for the displayed target presentation.

[0111] Optionally, based on the above device, the first presentation operation determination submodule includes:

[0112] The first presentation operation determination unit is used to determine the presentation operation of the target object for the displayed target presentation according to the action data when the action data meets the preset operation condition.

[0113] Optionally, based on the above device, the device further includes:

[0114] The object image display module is used to display the object image when the action data does not meet the preset operation conditions.

[0115] Optionally, based on the above device, the action data determination submodule includes:

[0116] An action recognition model acquisition unit, used to acquire an action recognition model;

[0117] The action data determination unit is used to determine the action data of the target object according to the object image and the action recognition model.

[0118] Optionally, the device further comprises:

[0119] The parsing data obtaining module is used to obtain the voice data of the target object and parse the voice data to obtain parsed data;

[0120] The presentation operation determination module 410 includes:

[0121] The second presentation operation determination submodule is used to determine the presentation operation of the target object with respect to the displayed target presentation according to the object image when the parsed data indicates that the target object has an operation action.

[0122] Optionally, based on the above device, the second presentation operation determination submodule includes:

[0123] The second presentation operation determination unit is used to determine the presentation operation of the target object for the displayed target presentation according to the object image and the parsed data.

[0124] The presentation operation device provided in the embodiment of the present invention obtains the object image of the target object through the presentation operation determination module, and determines the presentation operation of the target object for the displayed target presentation according to the object image, so as to determine the operation required for the displayed target presentation; and displays the object image through the target presentation operation module, and synchronously operates the target presentation according to the presentation operation, so as to display the object image and operate the presentation. The above device first determines the presentation operation after obtaining the object image, then displays the object image, and synchronously operates the target presentation according to the presentation operation, so as to avoid the problem that the object image of the target object for operating the presentation and the operation on the presentation are out of sync due to the independence of displaying the object image and determining the presentation operation, and realizes the synchronization of displaying the object image and operating the presentation.

[0125] The presentation operation device provided in the embodiment of the present invention can execute the presentation operation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0126] It is worth noting that in the embodiment of the above-mentioned presentation operation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0127] Figure 6A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0128] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0129] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the presentation operation method.

[0131] In some embodiments, the presentation operation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the presentation operation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the presentation operation method in any other appropriate manner (e.g., by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0138] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0139] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A presentation operation method, characterized in that: include: Acquire an object image of a target object, and determine, based on the object image, a presentation operation of the target object with respect to a displayed target presentation; The object image is displayed, and the target presentation is operated synchronously according to the presentation operation.

2. The method according to claim 1, characterized in that: The step of determining, according to the object image, a presentation operation of the target object with respect to the displayed target presentation includes: Determining motion data of the target object according to the object image; The presentation operation of the target object on the displayed target presentation is determined according to the action data.

3. The method according to claim 2, characterized in that The step of determining the presentation operation of the target object on the displayed target presentation according to the action data includes: Determining an operation gesture of the target object according to the action data; From at least one candidate operation, determine an alternative operation having a mapping relationship with the operation gesture as a presentation operation of the target object for the displayed target presentation.

4. The method according to claim 3, characterized in that Also includes: determining a body movement of the target object according to the movement data; The determining, from at least one candidate operation, an alternative operation having a mapping relationship with the operation gesture as a presentation operation of the target object for the displayed target presentation includes: From at least one candidate operation, determine an alternative operation having a mapping relationship with the operation gesture and the body movement as a presentation operation of the target object for the displayed target presentation.

5. The method according to claim 2, characterized in that: The step of determining the presentation operation of the target object on the displayed target presentation according to the action data includes: In the case where the action data meets the preset operation condition, the presentation operation of the target object on the displayed target presentation is determined according to the action data.

6. The method according to claim 5, characterized in that Also includes: When the action data does not satisfy the preset operation condition, the object image is displayed.

7. The method according to claim 2, characterized in that The step of determining the motion data of the target object according to the object image includes: Get the action recognition model; The motion data of the target object is determined according to the object image and the motion recognition model.

8. The method according to claim 1, characterized in that Also includes: Acquire the voice data of the target object, and parse the voice data to obtain parsed data; The step of determining, according to the object image, a presentation operation of the target object with respect to the displayed target presentation includes: In a case where the parsed data indicates that an operation action has been performed on the target object, a presentation operation of the target object on the displayed target presentation is determined based on the object image.

9. The method according to claim 8, characterized in that The step of determining, according to the object image, a presentation operation of the target object with respect to the displayed target presentation includes: A presentation operation of the target object with respect to the displayed target presentation is determined according to the object image and the parsed data.

10. A presentation operation device, characterized in that: include: A presentation operation determination module, used to obtain an object image of a target object, and determine a presentation operation of the target object with respect to a displayed target presentation according to the object image; The target presentation operation module is used to display the object image and synchronously operate the target presentation according to the presentation operation.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the presentation operation method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the presentation operation method as described in any one of claims 1-9 when executed.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the presentation operation method according to any one of claims 1 to 9.

Images