Interactive identification method, identification system, computer device and storage medium

Abstract

The present invention discloses an interactive identification method, an interactive identification system, a computer device and a storage medium. The interactive identification method comprises: controlling an image acquisition device to collect scene images of an activity room in real time at a predetermined frequency within a predetermined time; performing posture recognition on each participant in the scene image, judging whether each participant makes at least one of a plurality of preset postures, and if so, storing the posture information of the posture; if the posture information meets a preset condition, judging whether the participant making the current posture has a preset interactive behavior, and if so, recording the interactive behavior. The embodiment provided by the present invention can recognize the interactive behavior of each participant in a classroom or a meeting in real time, thereby effectively evaluating the activity of each participant and the interactivity of the classroom or meeting.

Description

Technical Field

[0001] The present invention relates to the technical field of gesture recognition, and in particular to an interactive recognition method, an interactive recognition system, a computer device and a storage medium based on gesture recognition. Background Art

[0002] In traditional classroom teaching, the analysis and observation of students' classroom interaction behaviors are mostly subjective evaluations by teaching assistants, and the results may not accurately reflect the actual classroom interaction situation, and the statistics of classroom interaction will greatly consume the energy of teaching assistants. Based on such problems, the educational whiteboard system focuses on statistics and analysis of students' attention, interactivity and other behaviors in the classroom, while the existing educational whiteboard products mostly detect students' interactive behaviors through simple action detection, and the diversity of students' classroom actions or false actions will greatly affect the accuracy and reliability of the statistical results. Summary of the invention

[0003] In order to solve at least one of the above problems, a first embodiment of the present invention provides an interactive recognition method based on gesture recognition, comprising:

[0004] Control the image acquisition device to acquire scene images of the activity room in real time at a predetermined frequency within a predetermined time;

[0005] Performing posture recognition on each participant in the scene image to determine whether each participant has made at least one of a plurality of preset postures, and if so, storing posture information of the posture;

[0006] If the gesture information meets the preset conditions, it is determined whether the participant making the current gesture has a preset interactive behavior, and if so, the interactive behavior is recorded.

[0007] Furthermore, if the gesture satisfies a preset condition, determining whether the participant making the current gesture has a preset interactive behavior, and if so, recording the interactive behavior further includes:

[0008] If the current posture is the first preset posture, the display device is controlled to display the preset manual posture request, and in response to the operation of the first preset participant, it is determined whether the participant making the current posture has a preset interactive behavior, and if so, the interactive behavior is recorded.

[0009] Furthermore, if the gesture satisfies a preset condition, determining whether the participant making the current gesture has a preset interactive behavior, and if so, recording the interactive behavior further includes:

[0010] If the current posture is the first preset posture, it is determined whether the participant of the current posture has a preset interactive behavior according to the posture information stored before the posture occurrence time of the current posture, and if so, the interactive behavior is recorded.

[0011] Furthermore, the gesture information includes: gesture type, gesture occurrence time, and location information of the participant who made the gesture;

[0012] If the current posture is the first preset posture, judging whether the participant of the current posture has a preset interactive behavior according to the posture information stored before the posture occurrence time of the current posture, and if so, recording the interactive behavior further includes:

[0013] If the current posture is the first preset posture, it is determined whether the participant making the current posture has a previous posture before the posture occurrence time of the current posture, and whether the previous posture is the second preset posture,

[0014] If the previous posture is a second preset posture, determining whether the time difference between the posture occurrence time of the previous posture and the posture occurrence time of the current posture meets the first time threshold, and if so, determining that the participant of the current posture has a preset interactive behavior and recording the interactive behavior;

[0015] Otherwise, it is determined whether the first preset participant has made the third preset gesture.

[0016] Furthermore, the posture information of the first preset participant also includes a posture direction;

[0017] Otherwise, determining whether the first preset participant has made a third preset gesture further includes:

[0018] Determine whether the first preset participant has a previous posture before the posture occurrence time of the current posture, and whether the previous posture of the first preset participant is a third preset posture,

[0019] If the previous gesture of the first preset participant is the third preset gesture, then determine whether the time difference between the gesture occurrence time of the previous gesture of the first preset participant and the gesture occurrence time of the current gesture meets the second time threshold; if so, determine whether the position information of the participant making the current gesture meets the gesture direction of the previous gesture of the first preset participant; if so, determine that the participant in the current gesture has a preset interactive behavior and record the interactive behavior;

[0020] Otherwise, the display device is controlled to display a preset manual confirmation request, and in response to the operation of the first preset participant, it is determined whether the participant making the current gesture has a preset interactive behavior, and if so, the interactive behavior is recorded.

[0021] Furthermore, before the image acquisition device is controlled to acquire the scene image of the activity room in real time at a predetermined frequency within a predetermined time, the interactive recognition method further includes:

[0022] Determine whether the location binding is completed. If not, control the display device to display a preset manual binding request, and in response to the operation of the first preset participant, bind the identity information and location information of each participant and confirm that each participant is already at the preset location.

[0023] Furthermore, before determining whether the location binding is completed, if not, controlling the display device to display a preset manual binding request, and responding to the operation of the first preset participant to bind the identity information of each participant with the location information and confirming that each participant is already at the preset location, the interactive recognition method further includes:

[0024] It is determined whether the position setting is completed, and if not, the display device is controlled to display a preset manual setting request, and the position of the activity room is set in response to the operation of the first preset participant.

[0025] Further, the activity room is a classroom, the first preset participant is a teacher, the other participants are students, the first preset gesture is that the student stands up, the second preset gesture is that the student raises his hand, the third preset gesture is that the teacher extends his hand to call the name, and the interactive behavior is answering a question;

[0026] or

[0027] The activity room is a conference room, the first preset participant is the host, the other participants are participants, the first preset posture is the participant standing up, the second preset posture is the participant raising his hand, the third preset posture is the host extending his hand to call a name, and the interactive behavior is discussing the issue.

[0028] A second embodiment of the present invention provides an interactive recognition system based on gesture recognition, comprising a controller and an image acquisition device, wherein

[0029] The image acquisition device is used to acquire scene images of the activity room;

[0030] The controller is used to identify the posture of each participant in the scene image and record the posture information, determine whether the current posture meets the preset conditions, and if so, determine whether the participant making the current posture has a preset interactive behavior, and if so, record the interactive behavior.

[0031] Furthermore, it also includes a display device for displaying a plurality of preset manual requests and displaying the posture information of each of the postures.

[0032] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0033] A fourth embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0034] The beneficial effects of the present invention are as follows:

[0035] In view of the existing problems, the present invention develops an interactive recognition method, an interactive recognition system, a computer device and a storage medium. An embodiment of the present application uses a controller to perform posture recognition on scene images in an activity room collected in real time by an image acquisition device to identify interactive behaviors in the scene, so as to record the interactive behaviors in the scene, so as to facilitate statistics, analysis and evaluation of the activity level of each participant in the scene and the interactivity in the scene. At the same time, an educational whiteboard is used to provide display functions and manual operations to further improve the accuracy and reliability of the interactive recognition method, thereby making up for the problems in the prior art and having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A flow chart showing an interactive identification method according to an embodiment of the present invention;

[0038] Figure 2 A schematic diagram showing a teaching whiteboard according to an embodiment of the present invention;

[0039] Figure 3 A schematic diagram showing the structure of an interactive identification system according to an embodiment of the present invention;

[0040] Figure 4 A schematic structural diagram of a computer device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0041] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0042] like Figure 1 As shown, an embodiment of the present invention provides an interactive recognition method based on gesture recognition, comprising: controlling an image acquisition device to acquire a scene image of an activity room in real time at a predetermined frequency within a predetermined time; performing gesture recognition on each participant in the scene image, and determining whether each participant makes at least one of a plurality of preset gestures, and if so, storing the gesture information of the gesture; if the gesture information meets a preset condition, determining whether the participant making the current gesture has a preset interactive behavior, and if so, recording the interactive behavior.

[0043] In a specific example, the interactive behavior between a teacher and students in a classroom is used as an example to illustrate. After the teacher asks a question, the student's hand raising, the teacher's hand calling, and the student's standing up gestures are identified to determine whether the student and the teacher are interacting to answer the question.

[0044] Specific:

[0045] First, in response to the control of the controller, the image acquisition device acquires scene images in the classroom in real time at a predetermined frequency and sends the scene images to the controller.

[0046] In the classroom interaction scene of this embodiment, the controller uses the time difference between the class time and the end of get out of class time as the interaction recognition time, and controls the image acquisition device to capture the scene image in the classroom in real time at a frequency of taking a picture every 0.1 seconds within this predetermined time. In this embodiment, the controller and the image acquisition device can be an integrated structure or a separate structure. The image acquisition device can be a camera, an infrared camera, or a depth camera, etc., which has a camera function. This application does not limit the number of image acquisition devices, so that all students and teachers in the classroom can be captured.

[0047] Secondly, the controller performs image processing on the scene image, identifies each student and teacher in the image based on the portrait recognition technology, and then performs posture recognition on each student and teacher in the image according to a preset posture, and stores the recognized posture.

[0048] In the classroom interaction scene of the present embodiment, the student's "hand-raising" gesture, the teacher's "hand-raising" gesture and the student's "standing up" gesture are preset according to the interactive behavior of answering questions. For example, the "hand-raising" gesture recognition is performed on the student, and when it is recognized that there is a student with a "hand-raising" gesture, the gesture is recorded, specifically including the gesture type (hand-raising), the gesture occurrence time, and the seat of the student who made the gesture. For another example, the "hand-raising" gesture recognition is performed on the teacher, and when it is recognized that the teacher has a "hand-raising" gesture, the gesture is recorded, and the gesture information also includes the gesture direction, specifically including the gesture type (hand-raising), the gesture occurrence time, the teacher's location, and the gesture direction (the direction of the teacher's hand-raising gesture). And, the "standing up" gesture recognition is performed on the student, and when it is recognized that there is a student with a "standing up" gesture, the gesture is recorded, specifically including the gesture type (standing up), the gesture occurrence time, and the seat of the student who made the gesture.

[0049] In order to improve the accuracy of the controller in recognizing the posture of each student or teacher, in an optional embodiment, before class, the controller determines whether the position of the activity room has been set. If not, the controller controls the display device to display a manual setting request, and sets the position of the activity room in response to the operation of the first preset participant.

[0050] In this embodiment, before class, the controller determines whether the positions in the classroom have been pre-set, such as the position of the podium, the position layout of the student seats, etc. If not, a manual setting request is displayed through the display device to prompt the teacher to set the position information in the classroom. In this embodiment, the display device can be a device with a display function installed in the classroom, such as a teaching whiteboard. The teaching whiteboard can not only be used to display courseware information and prompt information, but also to display the posture information of each posture identified above. For example, if the hand-raising gestures of multiple students are recognized, the position information of the students who raised their hands will be displayed on the teaching whiteboard, etc. Figure 2 As shown, the teaching whiteboard 1 can also integrate an image acquisition device 2 and a controller (not shown in the figure), as well as a courseware display area 3 and a posture information display area 4. In this embodiment, the controller displays prompt information in the courseware display area to prompt the teacher to set the podium position, student seats, etc. in the classroom so that the subsequent controller can recognize the scene image in the classroom, thereby improving the accuracy of posture recognition of each student and teacher in the classroom.

[0051] In order to further improve the accuracy of the controller in recognizing the posture of each student or teacher, in another optional embodiment, before the course starts, the interactive recognition method further includes: determining whether the position binding is completed, if not, controlling the display device to display a preset manual binding request, and in response to the operation of the first preset participant, binding the identity information of each participant with the position information and confirming that each participant is already at the preset position

[0052] In this embodiment, before the course starts, the controller also determines whether the location binding is completed, that is, whether the student information, teacher information and specific location information have been bound. If not, a manual binding request is displayed through the display device to prompt the teacher to enter the teacher information and student information of this course. Binding the student information and the specific location information specifically includes binding the student's identity information, such as the student's name, student number and other identity information, with the location of the student's seat. Specifically, the student's name is Wang Yi, the student number is 2, and the location is the third row and second column.

[0053] Before the class starts, the controller determines whether the position is bound. If not, the teacher is prompted to set up the binding, such as binding the students attending the class with the position information of their seats, or modifying the original binding information to ensure that all students in attendance have taken their seats. For students who are on leave or absent, the teacher does not need to perform binding, thereby further enhancing the controller's gesture recognition performance for participating students based on effective positions, thereby improving the accuracy of gesture recognition. At the same time, the position binding result of this step can also be used as the student's attendance record.

[0054] Finally, the controller determines the current posture. If the current state is the student's "standing up" posture, it determines whether the standing student has completed an interactive behavior of answering the question. If so, the interactive behavior is recorded.

[0055] This embodiment performs posture recognition on the scene image according to preset postures, and records the posture information of the teacher and each student in the classroom respectively. When it is recognized that the student makes a "standing up" gesture, it is determined whether the standing student completes an interactive behavior of answering a question. If completed, the interactive behavior is recorded to facilitate statistics, analysis, and evaluation of the activity level of each participant in the classroom, as well as the interactivity of the class or meeting.

[0056] In the classroom interaction scenario of this embodiment, considering that in the interactive behavior of answering questions, there is a high probability that many students will raise their hands to answer the question after the teacher asks a question, or the teacher extends his hand to call on one of the students who raised his hand or a student who did not raise his hand, identifying the student's "standing up" posture is used as the basis for judging whether the student has completed an interactive behavior of answering the question.

[0057] Specifically, in an optional embodiment, if the current posture is a first preset posture, the display device is controlled to display a preset manual posture request, and in response to the operation of the first preset participant, it is determined whether the participant who makes the current posture has a preset interactive behavior, and if so, the interactive behavior is recorded.

[0058] In this embodiment, to avoid errors in recognition, when a student stands up to answer a question, the controller controls the teaching whiteboard to display a gesture confirmation request to prompt the teacher, who then manually confirms whether the student who stood up has completed an interactive behavior of answering the question.

[0059] Although manual confirmation by the teacher can ensure the accuracy of interactive behavior recognition, this method has the problem of frequently interrupting the teacher's lecture. In an optional embodiment, if the current posture is the first preset posture, the posture information stored before the posture occurrence time of the current posture is used to determine whether the participant of the current posture has a preset interactive behavior, and if so, the interactive behavior is recorded.

[0060] In this embodiment, since the image collector collects scene images in the classroom in real time, the controller identifies the postures of students "raising their hands", teachers "calling their names" and students "standing up" based on the scene images. Therefore, it is proposed that the controller comprehensively judge whether the student who "stands up" has completed the interactive behavior of answering the question based on whether there is a "raising hand" gesture before the student "stands up" and whether there is a "teacher's "calling their names" gesture before the student "stands up".

[0061] In an optional embodiment, it specifically includes: the posture information includes: posture type, posture occurrence time, and location information of the participant who makes the posture; if the current posture is the first preset posture, then judging whether the participant of the current posture has a preset interactive behavior based on the posture information stored before the posture occurrence time of the current posture, and if so, recording the interactive behavior further includes: if the current posture is the first preset posture, then judging whether the participant who makes the current posture has a previous posture before the posture occurrence time of the current posture, and whether the previous posture is a second preset posture, if the previous posture is the second preset posture, then judging whether the time difference between the posture occurrence time of the previous posture and the posture occurrence time of the current posture meets the first time threshold, if so, judging that the participant of the current posture has a preset interactive behavior and recording the interactive behavior; otherwise, judging whether the first preset participant has made a third preset posture.

[0062] In this embodiment, the controller queries the identity information of the student according to the position information of the "standing up" student, and queries whether the student has a "raising hand" gesture before "standing up" according to the student's identity information. If there is a "raising hand" gesture, it determines whether the difference between the occurrence time of the "raising hand" gesture and the current "standing up" gesture is less than 30 seconds. If it is less than 30 seconds, it is considered that the student's "raising hand" gesture and "standing up" gesture are successive gestures in an interactive behavior, thereby determining that the student has completed an interactive behavior of answering a question. The controller records the interactive behavior to facilitate the evaluation of the student's participation level in this course and the interactive situation of this course.

[0063] Otherwise, if the time difference between the previous "raising hand" gesture and the subsequent "standing up" gesture is greater than 30 seconds, the two gestures are judged to be independent of each other and cannot be used to determine whether the student has completed answering the question. In this case, it is also necessary to determine the teacher's gesture before the student "stands up".

[0064] In an optional embodiment, the posture information of the first preset participant also includes a posture direction; it is determined whether the first preset participant has a previous posture before the posture occurrence time of the current posture, and whether the previous posture of the first preset participant is a third preset posture; if the previous posture of the first preset participant is the third preset posture, it is determined whether the time difference between the posture occurrence time of the previous posture of the first preset participant and the posture occurrence time of the current posture meets the second time threshold; if so, it is determined whether the position information of the participant making the current posture meets the posture direction of the previous posture of the first preset participant; if so, it is determined that the participant in the current posture has a preset interactive behavior and the interactive behavior is recorded; otherwise, the display device is controlled to display a preset manual confirmation request, and in response to the operation of the first preset participant, it is determined whether the participant making the current posture has a preset interactive behavior; if so, the interactive behavior is recorded.

[0065] In this embodiment, it is first determined whether the teacher has the gesture of "stretching out his hand and calling the name" before the student "stands up". If the teacher has this gesture, it is determined whether the difference between the occurrence time of the "stretching out his hand and calling the name" gesture and the occurrence time of the current "standing up" gesture is less than 10 seconds. If it is less than 10 seconds, it is considered that the teacher's "stretching out his hand and calling the name" gesture and the current student's "standing up" gesture are successive gestures in an interactive behavior. At the same time, it is determined whether the gesture direction in the gesture information of the teacher's "stretching out his hand and calling the name" gesture, that is, whether the direction of the teacher's hand stretching out his hand and calling the name is the same as the position of the "standing up" student. If they are the same, it is determined that the student has completed an interactive behavior of answering a question. The controller records the interactive behavior to facilitate the evaluation of the student's participation in this course and the interaction of this course.

[0066] Otherwise, if the time between the teacher's "reaching out and calling the name" gesture and the student's "standing up" gesture is greater than 10 seconds or the direction of the teacher's "reaching out and calling the name" gesture is different from the position of the "standing up" student, then it is impossible to judge whether the current "standing up" student has completed the interactive operation of answering the question based on these two gestures. The controller controls the teaching whiteboard to display a manual confirmation request to prompt the teacher to manually confirm whether the current "standing up" student has completed the interactive operation of answering the question.

[0067] It is worth noting that this embodiment is only used to illustrate a specific implementation of the present application, and does not limit the application scenario of the present application. The present application can also be applied to other scenarios with interactive behaviors, for example, the activity room is a conference room, the first preset participant is the host, the other participants are participants, the first preset gesture is the participant standing up, the second preset gesture is the participant raising his hand, the third preset gesture is the host extending his hand to call the name, and the interactive behavior is discussing the issue. The specific implementation is similar to the above embodiment and will not be repeated here.

[0068] Corresponding to the interactive identification method provided in the above-mentioned embodiments, an embodiment of the present application also provides an interactive identification system based on posture recognition. Since the interactive identification system provided in the embodiment of the present application corresponds to the interactive identification methods provided in the above-mentioned embodiments, the previous implementation method is also applicable to the interactive identification system provided in this embodiment and will not be described in detail in this embodiment.

[0069] like Figure 3 As shown, an embodiment of the present application also provides an interactive recognition system based on gesture recognition, including a controller and an image acquisition device, wherein the image acquisition device is used to acquire a scene image of the activity room to which it belongs; the controller is used to perform gesture recognition on each participant in the scene image and record gesture information, and determine whether the current gesture meets a preset condition. If so, determine whether the participant making the current gesture has a preset interactive behavior, and if so, record the interactive behavior.

[0070] In this embodiment, the controller identifies each participant in the scene image captured by the image acquisition device, and recognizes the posture of each participant, thereby judging whether each participant has interactive behavior based on the posture that meets preset conditions. If so, the interactive behavior is recorded to facilitate statistics, analysis, and evaluation of the interactivity and participation of each participant in the application scenario. Otherwise, the identification continues until the activity of the application scenario ends.

[0071] In an optional embodiment, the interactive recognition system further includes a display device for displaying a plurality of preset manual requests and displaying posture information of each of the postures.

[0072] In this embodiment, the display device is a teaching whiteboard, such as Figure 2 As shown, the teaching whiteboard 1 can also integrate an image acquisition device 2 and a controller (not shown in the figure), as well as a courseware display area 3 and a posture information display area 4. The teaching whiteboard can not only be used to display courseware information and display a variety of pre-set manual requests, such as prompt information such as "manual confirmation request", "manual binding request" and "manual setting request" in the above embodiment, but also display the posture information of each posture identified above, for example, if the hand-raising gestures of multiple students are identified, the position information and identity information of the students who raised their hands will be displayed on the teaching whiteboard.

[0073] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements: controlling an image acquisition device to acquire scene images of the activity room in real time at a predetermined frequency within a predetermined time; performing posture recognition on each participant in the scene image to determine whether each participant makes at least one of a plurality of preset postures, and if so, storing posture information of the posture; if the posture information meets preset conditions, determining whether the participant making the current posture has a preset interactive behavior, and if so, recording the interactive behavior.

[0074] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0075] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0076] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0077] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] like Figure 4 As shown, a structural schematic diagram of a computer device provided by another embodiment of the present invention. Figure 4 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0079] like Figure 4 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0080] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0081] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0082] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0083] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0084] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 4 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0085] The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing an interactive recognition method based on gesture recognition provided by an embodiment of the present invention.

[0086] In view of the existing problems, the present invention develops an interactive recognition method, an interactive recognition system, a computer device and a storage medium. An embodiment of the present application uses a controller to perform posture recognition on scene images in an activity room collected in real time by an image acquisition device to identify interactive behaviors in the scene, so as to record the interactive behaviors in the scene, so as to facilitate statistics, analysis and evaluation of the activity level of each participant in the scene, as well as the interactivity of the classroom or meeting. At the same time, an educational whiteboard is used to provide display functions and manual operations to further improve the accuracy and reliability of the interactive recognition method, thereby making up for the problems in the prior art and having broad application prospects.

[0087] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. An interactive recognition method based on gesture recognition, characterized in that: include: Control the image acquisition device to acquire scene images of the activity room in real time at a predetermined frequency within a predetermined time; Performing posture recognition on each participant in the scene image to determine whether each participant has made at least one of a plurality of preset postures, and if so, storing posture information of the posture; If the gesture information meets the preset conditions, it is determined whether the participant who makes the current gesture has a preset interactive behavior. If so, the interactive behavior is recorded. The gesture information includes: gesture type, gesture occurrence time, and location information of the participant who makes the gesture, and further includes: If the current posture is the first preset posture, it is determined whether the participant making the current posture has a previous posture before the posture occurrence time of the current posture, and whether the previous posture is the second preset posture, If the previous posture is a second preset posture, determining whether the time difference between the posture occurrence time of the previous posture and the posture occurrence time of the current posture meets the first time threshold, and if so, determining that the participant of the current posture has a preset interactive behavior and recording the interactive behavior; Otherwise, it is determined whether the first preset participant has made the third preset gesture.

2. The interactive identification method according to claim 1, characterized in that: If the gesture satisfies the preset condition, determining whether the participant making the current gesture has a preset interactive behavior, and if so, recording the interactive behavior further includes: If the current posture is the first preset posture, the display device is controlled to display the preset manual posture request, and in response to the operation of the first preset participant, it is determined whether the participant making the current posture has a preset interactive behavior, and if so, the interactive behavior is recorded.

3. The interactive identification method according to claim 1, characterized in that: If the gesture satisfies the preset condition, determining whether the participant making the current gesture has a preset interactive behavior, and if so, recording the interactive behavior further includes: If the current posture is the first preset posture, it is determined whether the participant of the current posture has a preset interactive behavior according to the posture information stored before the posture occurrence time of the current posture, and if so, the interactive behavior is recorded.

4. The interactive identification method according to claim 1, characterized in that: The posture information of the first preset participant also includes a posture direction; Otherwise, determining whether the first preset participant has made a third preset gesture further includes: Determine whether the first preset participant has a previous posture before the posture occurrence time of the current posture, and whether the previous posture of the first preset participant is a third preset posture, If the previous gesture of the first preset participant is the third preset gesture, then determine whether the time difference between the gesture occurrence time of the previous gesture of the first preset participant and the gesture occurrence time of the current gesture meets the second time threshold; if so, determine whether the position information of the participant making the current gesture meets the gesture direction of the previous gesture of the first preset participant; if so, determine that the participant in the current gesture has a preset interactive behavior and record the interactive behavior; Otherwise, the display device is controlled to display a preset manual confirmation request, and in response to the operation of the first preset participant, it is determined whether the participant making the current gesture has a preset interactive behavior, and if so, the interactive behavior is recorded.

5. The interactive identification method according to any one of claims 1 to 4, characterized in that: Before the image acquisition device is controlled to acquire the scene image of the activity room in real time at a predetermined frequency within a predetermined time, the interactive recognition method further includes: Determine whether the location binding is completed. If not, control the display device to display a preset manual binding request, and in response to the operation of the first preset participant, bind the identity information and location information of each participant and confirm that each participant is already at the preset location.

6. The interactive identification method according to claim 5, characterized in that: Before determining whether the location binding is completed, if not, controlling the display device to display a preset manual binding request, and binding the identity information of each participant with the location information in response to the operation of the first preset participant and confirming that each participant is already at the preset location, the interactive recognition method further includes: It is determined whether the position setting is completed, and if not, the display device is controlled to display a preset manual setting request, and the position of the activity room is set in response to the operation of the first preset participant.

7. The interactive identification method according to claim 1, characterized in that: The activity room is a classroom, the first preset participant is a teacher, the other participants are students, the first preset gesture is that the student stands up, the second preset gesture is that the student raises his hand, the third preset gesture is that the teacher extends his hand to call the name, and the interactive behavior is answering a question; or The activity room is a conference room, the first preset participant is the host, the other participants are participants, the first preset gesture is the participant standing up, the second preset gesture is the participant raising his hand, the third preset gesture is the host extending his hand to call a name, and the interactive behavior is discussing the issue.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images