Method and system for labeling PowerPoint based on gesture recognition

Abstract

The embodiment of the invention provides a method and a system for labeling a presentation file based on gesture recognition. The method is applied to the field of computer technology and manuscript control, and comprises the following steps: acquiring gesture data of a user in real time; performing screen calibration on the gesture data of the user to obtain a real-time gesture position of the user, and mapping the real-time gesture position of the user to the presentation file in real time; performing gesture recognition on the gesture data of the user, selectively performing response interaction on a gesture recognition result of the user and the annotation module of the presentation file, and obtaining gesture interaction feedback of the user; and based on the gesture interaction feedback of the user, executing a labeling operation matched with the gesture interaction feedback in the presentation file. In this way, the demonstration efficiency and interactivity of the speaker are improved, meanwhile, richer and more attractive demonstration content is brought to audiences, and the audiences and the speaker can obtain more efficient, more visual and more user-friendly PPT demonstration experience.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology and document control, and in particular to a method and system for marking presentation documents based on gesture recognition. Background Art

[0002] With the rapid development of information technology, presentations (PPT, PowerPoint) have become an indispensable tool for business meetings, education and training, product displays, etc. When presenting a presentation, speakers often need to switch pages and annotate content using a mouse or touch screen, which limits the interaction between the speaker and the audience and affects the fluency and efficiency of the presentation.

[0003] Products in the prior art that apply gesture recognition technology to PPT presentations generally have problems such as low recognition accuracy, complex operation, and poor real-time performance, which results in a poor user experience and cannot meet the diverse needs of users in actual presentations.

[0004] Therefore, there is an urgent need to develop a technical solution for PPT presentations that can accurately recognize complex gestures and achieve efficient and stable annotation. Summary of the invention

[0005] The present disclosure provides a method and system for annotating presentations based on gesture recognition, which solves the problems in the prior art that products that apply gesture recognition technology to PPT presentations generally have low recognition accuracy, complex operations, and poor real-time performance, which results in poor user experience and inability to meet the diverse needs of users in actual presentations. The present disclosure obtains user gesture data in real time and performs gesture recognition on the gesture data, and selectively performs response interaction with the gesture recognition result and the presentation annotation module to obtain user gesture interaction feedback, and then performs corresponding annotation operations on the presentation based on the gesture interaction feedback, thereby improving the speaker's presentation efficiency and interactivity, while providing the audience with richer and more attractive presentation content, so that the audience and the speaker can obtain a more efficient, more intuitive, and more user-friendly PPT presentation experience.

[0006] According to a first aspect of the present disclosure, a method for annotating a presentation based on gesture recognition is provided, comprising:

[0007] Get user gesture data in real time;

[0008] Perform screen calibration on the user's gesture data to obtain the user's real-time gesture position, and map the user's real-time gesture position on the presentation in real time;

[0009] Perform gesture recognition on the user's gesture data, and selectively perform response interaction between the user's gesture recognition result and the presentation annotation module to obtain the user's gesture interaction feedback;

[0010] And based on the user's gesture interaction feedback, a marking operation matching the gesture interaction feedback is performed in the presentation.

[0011] Optionally, before acquiring the user's gesture data in real time, the method further includes:

[0012] Obtaining preference data of each user, wherein the preference data includes at least one habitual gesture, an operation command corresponding to each habitual gesture, a preferred range value of recognition sensitivity of each gesture, at least one preferred color for marking, and a priority of each preferred color for marking;

[0013] Based on the preference data of each user, construct a preference database for each user;

[0014] Acquire the body data of the speaker, and determine the identity information of the speaker based on the body data of the speaker;

[0015] Based on the speaker's identity information, a preference database of users matching the speaker is retrieved.

[0016] Optionally, the performing screen calibration on the user's gesture data to obtain the user's real-time gesture position, and mapping the user's real-time gesture position on the presentation in real time includes:

[0017] Determine a gesture position of the user based on the gesture data of the user;

[0018] Calibrate the user's gesture position with the screen position of the presentation to obtain a calibration relationship between the user's gesture position and the screen position of the presentation;

[0019] Based on the calibration relationship between the user's gesture position and the screen position of the presentation, the user's real-time gesture position is mapped onto the presentation in real time.

[0020] Optionally, the performing gesture recognition on the user's gesture data, selectively performing a response interaction between the user's gesture recognition result and a presentation annotation module, and obtaining the user's gesture interaction feedback includes:

[0021] Based on the user's preference database, gesture recognition is performed on the current user's gesture data:

[0022] If the current user's gesture data matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then the user's gesture recognition result is determined to be the operation command, and the user's gesture recognition result is selectively interacted with the presentation annotation module to obtain the user's gesture interaction feedback;

[0023] Otherwise, the user's gesture recognition result is not executed to interact with the response of the presentation annotation module to obtain the user's gesture interaction feedback.

[0024] Optionally, if the gesture data of the current user matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the gesture data of the current user exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then determining that the gesture recognition result of the user is the operation command includes:

[0025] If the current user's gesture data matches the gesture data corresponding to the marking operation in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the marking operation in the user's preference database, then the user's gesture recognition result is determined to be a marking operation;

[0026] If it is recognized that the user's gesture data matches the gesture data corresponding to other operation commands in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the other operation commands in the user's preference database, then the user's gesture recognition result is determined to be the other operation command.

[0027] Optionally, the determining that the user's gesture recognition result is the operation command, and selectively performing a response interaction between the user's gesture recognition result and a presentation annotation module to obtain the user's gesture interaction feedback includes:

[0028] When the gesture recognition result of the user is a marking operation, the gesture recognition result of the user is executed and a response interaction is performed on the marking module of the presentation to obtain the gesture interaction feedback of the user.

[0029] Optionally, the step of performing a response interaction between the user's gesture recognition result and the annotation module of the presentation to obtain the user's gesture interaction feedback includes:

[0030] Re-acquire the user's gesture data;

[0031] Performing gesture recognition on the re-acquired user gesture data, and determining a marking mode and a marking type corresponding to the re-acquired user gesture data;

[0032] Acquire sensory data of the user, wherein the sensory data at least includes pressure data and trajectory data;

[0033] Based on the sensing data of the user, a gesture interaction feedback of the user is determined, wherein the gesture interaction feedback at least includes a pressure value and a movement trajectory.

[0034] Optionally, the step of performing a marking operation matching the gesture interaction feedback in the presentation based on the user's gesture interaction feedback includes:

[0035] Determining the annotation mode and annotation type executed by the annotation module of the presentation based on the annotation mode and annotation type corresponding to the re-acquired user gesture data;

[0036] Determine annotation parameters of the annotation operation based on the annotation mode and annotation type and the user's gesture interaction feedback, wherein the annotation parameters at least include the thickness of the annotation line and the annotation trajectory route;

[0037] The annotation operation is performed on the presentation based on the annotation parameters of the annotation operation to achieve annotation of the presentation.

[0038] Optionally, based on the user's gesture interaction feedback, after the presentation performs a marking operation matching the gesture interaction feedback, the method further includes:

[0039] Re-acquire the user's dynamic gesture data;

[0040] Performing gesture recognition on the dynamic gesture data, and selectively adjusting the annotation mode and annotation type executed by the annotation module of the presentation;

[0041] Based on the adjustment result, reacquiring the user's sensing data and subsequent steps are selectively performed.

[0042] According to a second aspect of the present disclosure, a system for annotating presentations based on gesture recognition is provided, comprising an acquisition unit, a presentation control unit and a control device, wherein the acquisition unit comprises an image acquisition module and a sensing module for acquiring gesture data of a user, body data of a speaker and sensing data of a user, the presentation control unit comprises at least an annotation module for performing an annotation operation on the presentation, and the control device comprises a processor and a memory, wherein the memory is suitable for storing a plurality of program codes, wherein the program codes are suitable for being loaded and run by the processor to execute the method for annotating presentations based on gesture recognition as described in any one of the technical solutions of the above-mentioned method for annotating presentations based on gesture recognition.

[0043] Compared with the prior art, the present invention has the following advantages or beneficial effects:

[0044] 1. Improve interactivity: By using gestures, speakers can communicate with the audience in a natural and intuitive way, which enhances the interactivity of the speech. In addition, the audience can intuitively understand the speaker's actions on the PPT, which increases the audience's participation and interest.

[0045] 2. Enhanced expressive ability: Gestures provide richer means of expression than traditional tools, allowing speakers to use various gestures to emphasize key points, depict shapes or simulate actions. Speakers can then convey emotions and emphasis through gestures, making the speech more vivid and convincing.

[0046] 3. Reduce operational interference: Speakers can annotate or perform other operations on presentations without frequent contact with computers or presentation equipment, which reduces interruptions and interference when operating equipment. This allows speakers to move freely in a larger space and establish closer connections with the audience. Speakers can focus on the content of the speech and the audience, rather than on equipment operation, thereby improving the concentration of both the audience and the speaker.

[0047] 4. Improve presentation efficiency: The presentation annotation system based on gesture recognition has a fast response capability, which allows the speaker's intentions to be converted into operations on the PPT in real time. During the presentation, it saves the speaker a lot of time and avoids the time cost of performing mouse operations or finding the correct buttons.

[0048] 5. Customizability: Speakers build their own preference database based on their preferences and presentation needs, and can then customize gesture commands to make the presentation more personalized.

[0049] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0051] Figure 1 A flowchart of a method for marking a presentation based on gesture recognition according to an embodiment of the present disclosure is shown;

[0052] Figure 2 A schematic diagram of the execution flow of a presentation annotation system based on gesture recognition according to an embodiment of the present disclosure is shown;

[0053] Figure 3 A block diagram of a system 200 for annotating presentations based on gesture recognition according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0055] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0056] In the present disclosure, the present disclosure obtains the user's gesture data in real time and performs gesture recognition on the gesture data, and selectively performs response interaction with the gesture recognition result and the annotation module of the presentation to obtain the user's gesture interaction feedback, and then performs corresponding annotation operations on the presentation according to the gesture interaction feedback, thereby improving the speaker's presentation efficiency and interactivity, and at the same time bringing richer and more attractive presentation content to the audience, so that the audience and the speaker can obtain a more efficient, more intuitive and more user-friendly PPT presentation experience.

[0057] Figure 1 FIG. 1 is a flow chart of a method 100 for annotating a presentation based on gesture recognition according to an embodiment of the present disclosure. Figure 1As shown, a method 100 for annotating a presentation based on gesture recognition includes:

[0058] S110: Acquire user's gesture data in real time;

[0059] Specifically, before acquiring the user's gesture data in real time, the method further includes:

[0060] Obtaining preference data of each user, wherein the preference data includes at least one habitual gesture, an operation command corresponding to each habitual gesture, a preferred range value of recognition sensitivity of each gesture, at least one preferred color for marking, and a priority of each preferred color for marking;

[0061] Based on the preference data of each user, construct a preference database for each user;

[0062] Acquire the body data of the speaker, and determine the identity information of the speaker based on the body data of the speaker;

[0063] Based on the speaker's identity information, a preference database of users matching the speaker is retrieved.

[0064] Specifically, the human body data can be video data or image data. The selection of human body data here is only an exemplary description. In actual testing, technical personnel in this field can make a selection according to actual needs, as long as the identity information of the speaker can be determined through the human body data. It will not be repeated here.

[0065] Specifically, the speaker's body data may be multiple types of face data, fingerprint data, and voice data. The determining of the speaker's identity information based on the speaker's body data includes:

[0066] Based on the body data of the speaker, it is determined that multiple data including the face data, fingerprint data, and voice data of the speaker match the preset identity information of the speaker.

[0067] Specifically, the speaker's identity information can be the speaker's identity code or the speaker's ID number. The selection of the speaker's identity information here is only an exemplary description. In actual testing, technical personnel in this field can make a selection based on actual needs, as long as it is possible to determine the user who matches the speaker through the speaker's identity information. No further details will be given here.

[0068] In the above embodiment, by obtaining the preference data of each user and building a preference database for each user, each user can set up the preference database according to personal preferences, thereby improving the user's usage experience, increasing the ease of operation, making it easier for users to understand and operate, and reducing the user's learning curve.

[0069] S120: performing screen calibration on the user's gesture data to obtain the user's real-time gesture position, and mapping the user's real-time gesture position on the presentation in real time;

[0070] Specifically, the screen calibration of the user's gesture data to obtain the user's real-time gesture position, and mapping the user's real-time gesture position on the presentation in real time includes:

[0071] Determine a gesture position of the user based on the gesture data of the user;

[0072] Calibrate the user's gesture position with the screen position of the presentation to obtain a calibration relationship between the user's gesture position and the screen position of the presentation;

[0073] Based on the calibration relationship between the user's gesture position and the screen position of the presentation, the user's real-time gesture position is mapped onto the presentation in real time.

[0074] Specifically, the user's gesture position is the three-dimensional coordinates of the user's gesture.

[0075] Specifically, before calibrating the user's gesture position with the screen position of the presentation to obtain a calibration relationship between the user's gesture position and the screen position of the presentation, the method further includes:

[0076] Obtaining the relative position and direction between the presentation screen and the system's image acquisition module;

[0077] Acquire the physical size and resolution of the screen, and acquire the acquisition angle of the image acquisition module;

[0078] Obtain the calibration parameters between the presentation's screen and the system's image acquisition module.

[0079] Specifically, calibrating the user's gesture position with the screen position of the presentation to obtain a calibration relationship between the user's gesture position and the screen position of the presentation includes:

[0080] Convert the three-dimensional coordinates of the user's gesture into two-dimensional coordinates of the user's gesture position on the presentation screen;

[0081] Determine mapping parameters of the user's gesture position on the presentation screen based on calibration parameters between the presentation screen and the image acquisition module of the system and the two-dimensional coordinates of the user's gesture position on the presentation screen;

[0082] The user's gesture position is mapped onto the screen of the presentation through a perspective transformation algorithm, so as to achieve a visual display of the calibration relationship between the user's gesture position and the screen position of the presentation.

[0083] In the above embodiment, the user's gesture position is determined through the user's gesture data, and the user's gesture position and the screen position of the presentation are calibrated, and the user's real-time gesture position is mapped on the presentation, so that the calibration relationship between the user's gesture position and the screen position of the presentation can be visualized, so that the audience can see the speaker's gestures directly converted into actions on the PPT, and the intuitive visual feedback can increase the audience's participation and interest.

[0084] S130: performing gesture recognition on the user's gesture data, and selectively performing response interaction between the user's gesture recognition result and the annotation module of the presentation, to obtain the user's gesture interaction feedback;

[0085] Specifically, the step of performing gesture recognition on the user's gesture data, selectively performing response interaction between the user's gesture recognition result and the annotation module of the presentation, and obtaining the user's gesture interaction feedback includes:

[0086] Based on the user's preference database, gesture recognition is performed on the current user's gesture data:

[0087] If the current user's gesture data matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then the user's gesture recognition result is determined to be the operation command, and the user's gesture recognition result is selectively interacted with the presentation annotation module to obtain the user's gesture interaction feedback;

[0088] Otherwise, the user's gesture recognition result is not executed to interact with the response of the presentation annotation module to obtain the user's gesture interaction feedback.

[0089] Specifically, if the gesture data of the current user matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the gesture data of the current user exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then determining that the gesture recognition result of the user is the operation command includes:

[0090] If the current user's gesture data matches the gesture data corresponding to the marking operation in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the marking operation in the user's preference database, then the user's gesture recognition result is determined to be a marking operation;

[0091] If it is recognized that the user's gesture data matches the gesture data corresponding to other operation commands in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the other operation commands in the user's preference database, then the user's gesture recognition result is determined to be the other operation command.

[0092] Specifically, the other operation commands may be PPT control commands for turning pages, zooming in and out. The selection of other operation commands here is only an exemplary description. In actual testing, technical personnel in this field can make a selection according to actual needs. As long as the corresponding PPT control command can be executed through the operation module corresponding to the operation command in the control unit of the system according to other operation commands, it will not be repeated here.

[0093] Specifically, determining that the user's gesture recognition result is the operation command, and selectively performing a response interaction between the user's gesture recognition result and a presentation annotation module to obtain the user's gesture interaction feedback includes:

[0094] When the gesture recognition result of the user is a marking operation, the gesture recognition result of the user is executed and a response interaction is performed on the marking module of the presentation to obtain the gesture interaction feedback of the user.

[0095] Specifically, determining that the user's gesture recognition result is the operation command, and selectively performing a response interaction between the user's gesture recognition result and a presentation annotation module to obtain the user's gesture interaction feedback also includes:

[0096] When the user's gesture recognition result is other operation commands, the user's gesture recognition result is not executed to interact with the presentation annotation module to perform a response to obtain the user's gesture interaction feedback;

[0097] The user's gesture recognition result is interacted with the operation module that performs the operation corresponding to the other operation command on the presentation to execute the control operation corresponding to the other operation command on the presentation.

[0098] Specifically, the step of performing a response interaction between the user's gesture recognition result and the annotation module of the presentation to obtain the user's gesture interaction feedback includes:

[0099] Re-acquire the user's gesture data;

[0100] Performing gesture recognition on the re-acquired user gesture data, and determining a marking mode and a marking type corresponding to the re-acquired user gesture data;

[0101] Acquire sensory data of the user, wherein the sensory data at least includes pressure data and trajectory data;

[0102] Based on the sensing data of the user, a gesture interaction feedback of the user is determined, wherein the gesture interaction feedback at least includes a pressure value and a movement trajectory.

[0103] Specifically, the annotation modes include at least annotation editing, annotation deletion, underline annotation, highlight annotation, shape annotation, and line annotation; the annotation types include at least editing position and editing content corresponding to annotation editing, deletion position and deletion content corresponding to annotation deletion, underline and strikethrough corresponding to underline annotation, area highlight and text highlight corresponding to highlight annotation, various graphics corresponding to shape annotation, and drawing tools corresponding to line annotation.

[0104] In the above embodiment, different annotation modes and annotation types are used to implement different annotation operations on the presentation using a variety of gesture annotations. In particular, by setting the annotation editing and annotation deletion functions, the speaker can conveniently undo and edit annotations, allowing the speaker to modify the annotations at any time during the presentation, thereby improving the accuracy of the annotations and ensuring that the presentation content always remains in the best state.

[0105] S140: Based on the user's gesture interaction feedback, perform a marking operation matching the gesture interaction feedback in the presentation.

[0106] Specifically, the performing of a marking operation matching the gesture interaction feedback in the presentation based on the user's gesture interaction feedback includes:

[0107] Determining the annotation mode and annotation type executed by the annotation module of the presentation based on the annotation mode and annotation type corresponding to the re-acquired user gesture data;

[0108] Determine annotation parameters of the annotation operation based on the annotation mode and annotation type and the user's gesture interaction feedback, wherein the annotation parameters at least include the thickness of the annotation line and the annotation trajectory route;

[0109] The annotation operation is performed on the presentation based on the annotation parameters of the annotation operation to achieve annotation of the presentation.

[0110] Specifically, based on the user's gesture interaction feedback, after the presentation performs a marking operation matching the gesture interaction feedback, the method further includes:

[0111] Re-acquire the user's dynamic gesture data;

[0112] Performing gesture recognition on the dynamic gesture data, and selectively adjusting the annotation mode and annotation type executed by the annotation module of the presentation;

[0113] Based on the adjustment result, reacquiring the user's sensing data and subsequent steps are selectively performed.

[0114] Specifically, after the annotation operation is performed on the presentation based on the annotation parameters of the annotation operation, the method further includes:

[0115] The annotation operations on the presentation are beautified and the beautified annotation operations are displayed.

[0116] Specifically, the beautification processing can be to use a smoothing algorithm (here using Bezier curves or spline curves as examples) to smooth the gesture trajectory, or to use a filtering algorithm (here using a Kalman filter as an example) to reduce gesture noise and improve line stability. It can also be line smoothing processing, intelligent selection of color and width to ensure the professionalism and beauty of the annotation. The choice of beautification processing here is only an exemplary explanation. In actual testing, technical personnel in this field can make a choice according to actual needs, as long as the annotation operation can be beautified to ensure the professionalism and beauty of the annotation, and it will not be repeated here.

[0117] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0118] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0119] Figure 3 A block diagram of a presentation annotation system based on gesture recognition according to an embodiment of the present disclosure is shown. Figure 3 As shown, the presentation annotation system 300 based on gesture recognition includes mainly a collection unit 302, a presentation control unit 303 and a control device 301, wherein the collection unit 302 includes an image acquisition module and a sensing module for acquiring the user's gesture data, the speaker's body data and the user's sensing data, and the presentation control unit 303 includes at least a marking module for performing an annotation operation on the presentation, and the control device 301 includes a processor 3011 and a memory 3012, and the memory 3012 can be configured to store a program code 3013 for executing the presentation annotation method based on gesture recognition of the above method embodiment, and the processor 3011 can be configured to execute the program code 3013 in the memory 3012, and the program code 3013 includes but is not limited to executing the presentation annotation method based on gesture recognition of the above method embodiment. For ease of explanation, only the part related to the embodiment of the present disclosure is shown, and if the specific technical details are not disclosed, please refer to the method part of the embodiment of the present disclosure. The control device 301 can be a control device 301 device formed by various electronic devices.

[0120] Specifically, the image acquisition module can be a depth camera device or a photo device. The selection of the image acquisition module here is only an exemplary explanation. In actual testing, technical personnel in this field can make a selection according to actual needs, as long as it is possible to obtain the user's gesture data and the speaker's face data in the body data through the image acquisition module. No further details will be given here.

[0121] Specifically, the sensing module can be a pressure sensor, a sound collection device, or a fingerprint collection device. The selection of the sensing module here is only an exemplary description. In actual testing, technical personnel in this field can make a selection according to actual needs, as long as it is possible to obtain the user's sensing data and the fingerprint data or sound data in the speaker's body data through the sensing module. No further details will be given here.

[0122] Specifically, the control unit 303 of the presentation also includes an operation module matching each other operation command, wherein the other operation commands include at least PPT control commands for page turning, zooming in and out, and the types of the operation modules include at least a page turning operation module, a zooming in operation module, and a zooming out operation module.

[0123] In one implementation, the description of the specific implementation functions may refer to steps S110 to S140.

[0124] The above-mentioned presentation annotation system based on gesture recognition is used to perform Figure 1 The embodiment of the method for marking a presentation based on gesture recognition is shown, and the execution Figure 2 The execution process of the system for marking presentations based on gesture recognition shown in the figure is similar in technical principles, technical problems solved and technical effects produced. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the system for marking presentations based on gesture recognition can refer to the contents described in the embodiment of the method for marking presentations based on gesture recognition, which will not be repeated here.

[0125] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present disclosure can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0126] Further, the system 300 for annotating presentations based on gesture recognition of the present disclosure also includes a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present disclosure, the computer-readable storage medium may be configured to store a program code 3013 for executing the method for annotating presentations based on gesture recognition of the above method embodiment, and the program code 3013 may be loaded and run by the processor 3011 to implement the above method for annotating presentations based on gesture recognition. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present disclosure. The computer-readable storage medium may be a memory 3012 device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present disclosure is a non-temporary computer-readable storage medium.

[0127] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present disclosure, the physical devices corresponding to these modules may be the processor 3011 itself, or a part of the software, a part of the hardware, or a part of the combination of software and hardware in the processor 3011. Therefore, the number of each module in the figure is only schematic.

[0128] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The program code 3013 for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes 3013 can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code 3013, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code 3013 can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0133] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0134] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principles of the present disclosure, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present disclosure.

[0135] So far, the technical solutions of the present disclosure have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present disclosure is obviously not limited to these specific embodiments. Without departing from the principles of the present disclosure, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present disclosure.

Claims

1. A method for annotating presentations based on gesture recognition, characterized in that: The method comprises: Get user gesture data in real time; Perform screen calibration on the user's gesture data to obtain the user's real-time gesture position, and map the user's real-time gesture position on the presentation in real time; Perform gesture recognition on the user's gesture data, and selectively perform response interaction between the user's gesture recognition result and the presentation annotation module to obtain the user's gesture interaction feedback; And based on the user's gesture interaction feedback, a marking operation matching the gesture interaction feedback is performed in the presentation.

2. The method according to claim 1, characterized in that Before acquiring the user's gesture data in real time, the method further includes: Obtaining preference data of each user, wherein the preference data includes at least one habitual gesture, an operation command corresponding to each habitual gesture, a preferred range value of recognition sensitivity of each gesture, at least one preferred color for marking, and a priority of each preferred color for marking; Based on the preference data of each user, construct a preference database for each user; Acquire the body data of the speaker, and determine the identity information of the speaker based on the body data of the speaker; Based on the speaker's identity information, a preference database of users matching the speaker is retrieved.

3. The method according to claim 2, characterized in that The screen calibration of the user's gesture data to obtain the user's real-time gesture position, and mapping the user's real-time gesture position on the presentation in real time includes: Determine a gesture position of the user based on the gesture data of the user; Calibrate the user's gesture position with the screen position of the presentation to obtain a calibration relationship between the user's gesture position and the screen position of the presentation; Based on the calibration relationship between the user's gesture position and the screen position of the presentation, the user's real-time gesture position is mapped onto the presentation in real time.

4. The method according to claim 3, characterized in that The step of performing gesture recognition on the user's gesture data, selectively performing response interaction between the user's gesture recognition result and the annotation module of the presentation, and obtaining the user's gesture interaction feedback includes: Based on the user's preference database, gesture recognition is performed on the current user's gesture data: If the current user's gesture data matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then the user's gesture recognition result is determined to be the operation command, and the user's gesture recognition result is selectively interacted with the presentation annotation module to obtain the user's gesture interaction feedback; Otherwise, the user's gesture recognition result is not executed to interact with the response of the presentation annotation module to obtain the user's gesture interaction feedback.

5. The method according to claim 4, characterized in that If the gesture data of the current user matches the gesture data corresponding to any operation command in the user's preference database, and the retention time of the gesture data of the current user exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the operation command in the user's preference database, then determining that the gesture recognition result of the user is the operation command includes: If the current user's gesture data matches the gesture data corresponding to the marking operation in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the marking operation in the user's preference database, then the user's gesture recognition result is determined to be a marking operation; If it is recognized that the user's gesture data matches the gesture data corresponding to other operation commands in the user's preference database, and the retention time of the current user's gesture data exceeds the preset recognition sensitivity preference range value corresponding to the gesture data corresponding to the other operation commands in the user's preference database, then the user's gesture recognition result is determined to be the other operation command.

6. The method according to claim 5, characterized in that The determining that the user's gesture recognition result is the operation command, and selectively performing a response interaction between the user's gesture recognition result and a presentation annotation module to obtain the user's gesture interaction feedback includes: When the gesture recognition result of the user is a marking operation, the gesture recognition result of the user is executed and a response interaction is performed on the marking module of the presentation to obtain the gesture interaction feedback of the user.

7. The method according to claim 6, characterized in that The step of performing a response interaction between the user's gesture recognition result and the annotation module of the presentation to obtain the user's gesture interaction feedback includes: Re-acquire the user's gesture data; Performing gesture recognition on the re-acquired user gesture data, and determining a marking mode and a marking type corresponding to the re-acquired user gesture data; Acquire sensory data of the user, wherein the sensory data at least includes pressure data and trajectory data; Based on the sensing data of the user, a gesture interaction feedback of the user is determined, wherein the gesture interaction feedback at least includes a pressure value and a movement trajectory.

8. The method according to claim 7, characterized in that The step of performing a marking operation matching the gesture interaction feedback in the presentation based on the user's gesture interaction feedback includes: Determining the annotation mode and annotation type executed by the annotation module of the presentation based on the annotation mode and annotation type corresponding to the re-acquired user gesture data; Determine annotation parameters of the annotation operation based on the annotation mode and annotation type and the user's gesture interaction feedback, wherein the annotation parameters at least include the thickness of the annotation line and the annotation trajectory route; The annotation operation is performed on the presentation based on the annotation parameters of the annotation operation to achieve annotation of the presentation.

9. The method according to claim 8, characterized in that Based on the user's gesture interaction feedback, after the presentation performs a marking operation matching the gesture interaction feedback, the method further includes: Re-acquire the user's dynamic gesture data; Performing gesture recognition on the dynamic gesture data, and selectively adjusting the annotation mode and annotation type executed by the annotation module of the presentation; Based on the adjustment result, reacquiring the user's sensing data and subsequent steps are selectively performed.

10. A presentation annotation system based on gesture recognition, characterized in that: The system includes an acquisition unit, a presentation control unit and a control device, the acquisition unit includes an image acquisition module and a sensing module, which are used to obtain user gesture data, speaker body data and user sensing data, the presentation control unit includes at least a labeling module, which is used to perform labeling operations on the presentation, and the control device includes a processor and a memory, the memory is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the presentation labeling method based on gesture recognition described in any one of claims 1 to 9.

Images