Teaching interaction behavior identification method and device, computer equipment and storage medium

Abstract

The invention relates to a teaching interaction behavior identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a behavior event flow in a teaching classroom; performing aggregation analysis on the behavior event flow to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event; determining an identification strategy corresponding to each composite teaching event according to the event type of each composite teaching event; identifying behavior events matched with the identification strategies in the behavior event flow by using the identification strategies corresponding to the composite teaching events; according to the method, the composite teaching events can be aggregated on the basis of the behavior events, and the effective interactive behaviors under the composite teaching events are sorted, so that the recognition accuracy of the classroom interactive behaviors is improved, and the misjudgment rate of the interactive behaviors is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a teaching interactive behavior recognition method, device, computer equipment and storage medium. Background Art

[0002] In classroom observation-type auxiliary teaching products, it is necessary to identify and count student interactive behaviors, such as identifying hand-raising behaviors, counting the number of hand-raising times, etc. The traditional method uses a camera to collect student images in real time and recognize students' hand-raising behaviors through visual detection. However, this method will mistakenly identify students' actions such as "stretching" as interactive behaviors. In actual classrooms, there are many interfering actions such as "stretching", resulting in a high error rate in identifying students' interactive behaviors. Summary of the invention

[0003] Based on this, it is necessary to provide a teaching interactive behavior recognition method, device, computer equipment and computer-readable storage medium that can screen effective interactive behaviors and improve the recognition accuracy of classroom interactive behaviors in response to the above-mentioned technical problems.

[0004] In a first aspect, the present application provides a method for identifying teaching interaction behaviors. The method comprises:

[0005] Obtain the behavioral event stream in the teaching classroom;

[0006] Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event;

[0007] According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event;

[0008] Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies;

[0009] The interactive behavior in the teaching class is determined based on the identified behavioral events.

[0010] In one embodiment, the aggregate analysis of the behavior event stream to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event includes:

[0011] Sequentially acquiring behavior events in the behavior event stream;

[0012] When the acquired behavior event triggers the transition condition of the current classroom state, the classroom state transition operation is executed;

[0013] The behavior event stream is aggregated into at least one composite teaching event based on the dynamically shifted classroom state, and the event type of each composite teaching event is determined.

[0014] In one embodiment, the process of determining whether the acquired behavior event triggers the transition condition of the current classroom state includes:

[0015] Obtaining a preset classroom state transition table, wherein the state transition table includes state transition conditions between various classroom states;

[0016] Matching the acquired behavior events with each state transition condition corresponding to the current classroom state in the state transition table to obtain a matching result;

[0017] It is determined based on the matching result whether the acquired behavior event triggers the transfer condition of the current classroom status.

[0018] In one embodiment, the classroom state based on the dynamic transfer aggregates the behavior event stream into at least one composite teaching event, and determines the event type of each composite teaching event, including:

[0019] Compare the dynamically transferred classroom status with the status conditions of each preset event type respectively;

[0020] When the dynamically transferred classroom state meets the state condition of the target event type, determining the duration of the classroom state that meets the state condition of the target event type;

[0021] According to the duration period, obtaining a segment event stream from the behavior event stream;

[0022] The acquired fragment event streams are aggregated into a composite teaching event, and the event type of the composite teaching event is determined to be the target event type.

[0023] In one embodiment, the step of using the identification strategies corresponding to the respective composite teaching events to identify the behavior events in the behavior event stream that match the identification strategies includes:

[0024] For each compound teaching event, determining at least one behavior event type according to the identification strategy corresponding to the current compound teaching event;

[0025] Searching according to the at least one behavior event type from the segment event stream corresponding to the current composite teaching event;

[0026] The searched behavior events are regarded as behavior events matching the identification strategy.

[0027] In one embodiment, the step of obtaining a behavior event stream in a teaching classroom includes:

[0028] Obtain video data and audio data in teaching classroom;

[0029] The video data and the audio data are analyzed to determine the flow of behavioral events in the teaching classroom.

[0030] In one embodiment, the method further comprises:

[0031] Upon receiving the interactive behavior display instruction, determining at least one teaching classroom corresponding to the interactive behavior display instruction;

[0032] Searching for corresponding interactive behaviors from the identification record file according to the at least one teaching classroom;

[0033] Classifying and counting the interactive behaviors according to the types of behaviors to obtain statistical results;

[0034] The statistical results are displayed.

[0035] In a second aspect, the present application also provides a teaching interactive behavior recognition device. The device comprises:

[0036] The acquisition module is used to obtain the behavior event stream in the teaching classroom;

[0037] An aggregation analysis module, used for performing aggregation analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event;

[0038] A determination module, used to determine the recording strategy corresponding to each composite teaching event according to the event type of each composite teaching event;

[0039] An identification module, used to use the identification strategies corresponding to each composite teaching event respectively to identify the behavior events matching the identification strategies in the behavior event stream;

[0040] The interactive behavior determination module is used to determine the interactive behavior in the teaching classroom according to the identified behavioral events.

[0041] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0042] Obtain the behavioral event stream in the teaching classroom;

[0043] Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event;

[0044] According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event;

[0045] Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies;

[0046] The interactive behavior in the teaching class is determined based on the identified behavioral events.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Obtain the behavioral event stream in the teaching classroom;

[0049] Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event;

[0050] According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event;

[0051] Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies;

[0052] The interactive behavior in the teaching classroom is determined based on the identified behavioral events.

[0053] The above-mentioned teaching interactive behavior recognition method, device, computer equipment and storage medium obtain the behavioral event stream in the teaching classroom; perform aggregation analysis on the behavioral event stream to obtain at least one compound teaching event in the teaching classroom and the event type of each compound teaching event; determine the recognition strategy corresponding to each compound teaching event according to the event type of each compound teaching event; use the recognition strategy corresponding to each compound teaching event respectively to identify the behavioral events matching the recognition strategy in the behavioral event stream; determine the interactive behavior in the teaching classroom according to the identified behavioral events, and can aggregate the compound teaching events based on the behavioral events, and sort out the effective interactive behaviors under the compound teaching events, thereby improving the recognition accuracy of classroom interactive behaviors and reducing the misjudgment rate of interactive behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A diagram showing an application environment of a teaching interaction behavior recognition method in an embodiment;

[0055] Figure 2 A schematic diagram of a flow chart of a teaching interaction behavior recognition method in one embodiment;

[0056] Figure 3 This is an example diagram of question-and-answer combination event aggregation in one embodiment;

[0057] Figure 4 is a schematic diagram of the process flow of an event in an embodiment;

[0058] Figure 5 A schematic diagram of a flow chart of a teaching interaction behavior identification method in another embodiment;

[0059] Figure 6 An example diagram of a behavior event sequence in one embodiment;

[0060] Figure 7 An example diagram of a classroom state transition table in one embodiment;

[0061] Figure 8 A schematic diagram of a process of classroom state transfer in an embodiment;

[0062] Fig. 9 A flowchart of a teaching interaction behavior recognition method in another embodiment;

[0063] Fig.10 is a structural block diagram of a teaching interactive behavior recognition device in an embodiment;

[0064] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] The teaching interaction behavior recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 collects multimedia information in the teaching classroom, and sends the collected multimedia information to the server 104 through the network for processing. The server 104 analyzes the behavioral event flow in the teaching classroom, aggregates compound teaching events based on the behavioral events, and sorts out effective interactive behaviors under the compound teaching events. Among them, the terminal 102 can be but is not limited to various cameras, microphones, computers and portable wearable devices, and the portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0067] In one embodiment, Figure 2 As shown, a teaching interaction behavior recognition method is provided. It can be understood that the method can also be applied to Figure 1 The server 104 in the embodiment can also be applied to the processing chip of the camera, and the method can be applied to Figure 1 The server 104 in the example is used as an example to illustrate, and the following steps are included:

[0068] Step 202, obtaining the behavior event stream in the teaching classroom.

[0069] Among them, a teaching class generally refers to a class from start to finish, and can also refer to at least one class from start to finish in a classroom within a certain period of time, which is not limited in this embodiment. A behavioral event describes a person in a teaching class completing a specific behavior at a certain point in time, such as: raising hands, standing up, asking questions, speaking, etc. A behavioral event stream refers to multiple behavioral events arranged in order of teaching class time. In some implementations, the behavioral event stream generated in the current teaching class is obtained in real time; in some implementations, the behavioral event stream occurring in a teaching class is determined by analyzing the multimedia information collected by a terminal in the teaching class.

[0070] Step 204: perform aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event.

[0071] Among them, a composite teaching event refers to an event corresponding to a teaching scene and composed of multiple behavioral events. The aggregation analysis conforms to certain aggregation rules. Behavioral events and composite teaching events can be abstracted as mathematical functions. The formula is: composite teaching event = f (behavioral event 1, behavioral event 2, ..., behavioral event N). Behavioral events are factors of the function. The specific logic of the function conforms to the aggregation rules, and the composite teaching event is the calculation result of the function. In some implementations, composite teaching events of different event types correspond to multiple mathematical functions. Multiple behavioral events that satisfy each mathematical function relationship in the behavioral event stream are aggregated into a composite teaching event. If multiple fragments contained in the behavioral event stream satisfy multiple mathematical function relationships respectively, multiple composite teaching events are aggregated, and the event type of each composite teaching event is determined.

[0072] In this embodiment, multiple event types are pre-set, and each event type corresponds to a composite teaching event, such as a question-and-answer scene event, a stage interaction event, etc., and each composite teaching event corresponds to a teaching scene. The following is an example to illustrate the aggregation analysis:

[0073] The compound teaching event is a question-and-answer scenario event. Referring to Table 1, the behavioral event factors corresponding to the question-and-answer scenario event include the teacher's question event, the student's hand-raising event, and the student's standing-up event. The aggregation rules include trigger conditions and continuous conditions.

[0074] Table 1:

[0075]

[0076] Assume that the behavior event flow of the teaching classroom is: e0, e0, e1, e2, e2, e3, e4, e5, e1. The aggregation rules shown in Table 1 show that in this behavior event flow, sub-streams e1, e2, e2, e3 meet the matching rules of the composite teaching event Q1. The sub-streams e1, e2, e2, e3 are extracted and aggregated into the composite teaching event Q1.

[0077] Reference Figure 3 ,The association relationships between behavioral events include spatial ,association, temporal association and dependent association. ,Spatial association refers to behavioral events occurring in the same classroom, temporal association refers to behavioral events within ,a sliding time window, and dependent association refers to the occurrence of event X ,depending on the occurrence of behavior Y. Figure 3 The behavioral events in are sorted according to the start time of the event, where a blank cell indicates that the corresponding behavioral event has not occurred in the corresponding time period, a solid cell indicates that the corresponding behavioral event has occurred in the corresponding time period, and a dotted box indicates a sliding time window. Through the association relationship between events and according to the aggregation rules of each compound teaching event, the behavioral events (teacher question event, student hand raising event, and student standing up event) are aggregated into a compound teaching event (question and answer scene event).

[0078] In some implementations, the server is provided with a state machine, and a classroom state transition table is used to manage the transition conditions between different classroom states. The state machine is used to initialize according to the classroom state transition table. After the initialization is completed, the state machine is used to trigger the state transfer according to the behavior event flow. During the state transfer process, a dynamic decision is made as to whether the behavior event flow can be aggregated into at least one composite teaching event, specific sub-flows are extracted and aggregated into composite teaching events, and the event type of the composite teaching event is determined.

[0079] Step 206: Determine the identification strategy corresponding to each composite teaching event according to the event type of each composite teaching event.

[0080] Among them, recognition strategies corresponding to compound teaching events of various event types are pre-set, and the compound teaching events corresponding to each event type have specific behavioral intentions. Recognition strategies are set by analyzing the behavioral intentions of each compound teaching event, which are used to identify valid behavioral events under each compound teaching event according to a specific recognition method. According to the event type of at least one compound teaching event aggregated in the behavioral event stream, the recognition strategy to be used is obtained from multiple pre-set recognition strategies.

[0081] Step 208: Use the identification strategies corresponding to the respective compound teaching events to identify the behavior events in the behavior event stream that match the identification strategies.

[0082] Among them, the obtained recognition strategy is used to identify valid behavior events from the sub-streams corresponding to each composite teaching event. Optionally, each recognition strategy corresponds to at least one behavior event type. For example, the behavior event types corresponding to the recognition strategy in the question-and-answer scenario event are hand-raising behavior events and standing up behavior events. Referring to Table 1, after the sub-streams e1, e2, e2, and e3 are aggregated into a composite teaching event Q1, it is reversely inferred that the behavior intentions of e2, e2, and e3 in Q1 are to answer the teacher's questions, and the behavior events e2, e2, and e3 are identified as valid behavior events in the composite teaching event Q1.

[0083] Step 210, determining the interactive behavior in the teaching classroom according to the identified behavioral events.

[0084] Among them, interactive behavior refers to the actions performed between teachers and students and between students in the teaching classroom to achieve teaching goals, complete teaching tasks, and realize information exchange. The process of determining interactive behavior based on the identified behavioral events can be to record all valid behavioral events in the teaching classroom as an interactive behavior, and record the number of occurrences of the interactive behavior. For example, two valid hand-raising behaviors and one standing behavior that occurred in the teaching classroom are recorded as three valid interactive behaviors. The process of determining interactive behavior based on the identified behavioral events can also be to record the valid behavioral events in the teaching classroom according to their types, for example, recording them as two hand-raising behaviors and one standing behavior, which is not limited in this embodiment.

[0085] In this embodiment, in the classroom teaching scenario, the behavior events are aggregated and analyzed according to certain rules to form a composite teaching event with business meaning that can be understood and accepted by users. Figure 4 , Figure 4 The following is a flow chart of an event processing, including:

[0086] Formatting step: formatting the visual analysis results and audio analysis results into behavioral events;

[0087] Preprocessing step: preprocessing behavioral events;

[0088] Pattern detection step: The behavioral events are aggregated and analyzed according to certain rules to form composite teaching events with business meanings that can be understood and accepted by users;

[0089] Event dispatching step: according to the specific type of compound teaching event, distribute it to the corresponding processing module;

[0090] Execute action steps: Execute different actions according to different compound event types, such as data statistics, summation, etc.

[0091] In the above-mentioned teaching interactive behavior identification method, the behavioral event stream in the teaching classroom is obtained; the behavioral event stream is aggregated and analyzed to obtain at least one compound teaching event in the teaching classroom and the event type of each compound teaching event; according to the event type of each compound teaching event, the identification strategy corresponding to each compound teaching event is determined; the identification strategy corresponding to each compound teaching event is used respectively to identify the behavioral events matching the identification strategy in the behavioral event stream; the interactive behavior in the teaching classroom is determined according to the identified behavioral events, and the compound teaching events can be aggregated on the basis of the behavioral events, and the effective interactive behaviors under the compound teaching events can be sorted out, thereby improving the recognition accuracy of classroom interactive behaviors and reducing the misjudgment rate of interactive behaviors.

[0092] In one embodiment, the step 202 includes: acquiring video data and audio data in the teaching classroom; analyzing the video data and the audio data to determine the behavioral event flow in the teaching classroom.

[0093] Among them, video data is collected by at least one camera installed in the classroom, and audio data is collected by at least one microphone installed in the classroom. The camera and the microphone can be integrated into an all-in-one machine to perform visual analysis on the video data collected by the camera to determine the behavioral events occurring at the physical level of different students at various time points, such as raising hands, standing up, etc. The audio data is analyzed to determine the behavioral events occurring at the language level of teachers and students at various time points, such as teachers asking questions, students speaking, etc. The analyzed behavioral events are arranged in chronological order to obtain a flow of behavioral events in the teaching classroom.

[0094] In one embodiment, the method also includes: when receiving an interactive behavior display instruction, determining at least one teaching classroom corresponding to the interactive behavior display instruction; searching for corresponding interactive behaviors from an identification record file based on the at least one teaching classroom; classifying and counting the interactive behaviors according to the behavior types to obtain statistical results; and displaying the statistical results.

[0095] Among them, after step 210, the interactive behaviors in the identified teaching classroom are stored in the identification record file. The interactive behaviors are displayed according to different teaching classrooms. In the specific implementation, the user can select at least one teaching classroom for query according to different statistical needs, for example, select the classroom in the same time period every day, select the classroom in the same classroom on the same day, select the classroom in all classrooms within a certain time period, select a specific classroom, input the interactive behavior display instruction to the server according to the statistical needs, and the server queries at least one teaching classroom targeted by the interactive behavior display instruction, classifies and counts the query results according to the behavior type, and obtains the statistical results. The behavior type refers to the specific type of interactive behavior, for example, raising hands, standing up, etc. The statistical results are as follows: raising hands A times, standing up B times, and going on stage C times. The statistical results are displayed to facilitate users to know the student interaction situation.

[0096] In one embodiment, Figure 5 As shown, step 204 includes:

[0097] Step 502: sequentially obtain the behavior events in the behavior event stream.

[0098] The individual behavior events in the behavior event stream are read sequentially in chronological order.

[0099] Step 504: When the acquired behavior event triggers the transition condition of the current classroom state, a classroom state transition operation is performed.

[0100] Among them, a state machine is used to trigger state transfer according to the behavioral events in the behavioral event flow. The state machine includes: current state, conditions, actions, and sub-states. The "current state" and "conditions" are causes, and the "actions" and "sub-states" are results. The current state refers to the current state of the classroom, that is, the current classroom state. The condition can be at least one behavioral event. When a behavioral event occurs, it triggers the execution of a state migration and / or triggers an action. An action refers to an action (optional) executed after the condition is met. After the action is executed, it can migrate to a new state or maintain the original state. The sub-state refers to the new state to be migrated to after the conditions are met. The "sub-state" is the next state relative to the "current state". When the "sub-state" is activated, it will serve as the "current state" at that moment.

[0101] The current classroom state is the current state of the state machine. Determine the transfer conditions and next state triggered by the behavioral events obtained in the current classroom state, execute the classroom state transfer operation, and transfer to the next state. Among them, executing the classroom state transfer operation can transfer to a new classroom state or maintain the original state.

[0102] Step 506: Aggregate the behavior event stream into at least one composite teaching event based on the dynamically shifted classroom state, and determine the event type of each composite teaching event.

[0103] The dynamically transferred classroom state refers to a classroom state that continuously changes during the dynamic transfer process by using a state machine to perform dynamic transfer under the triggering of each behavior event in the behavior event stream. In this embodiment, a state machine is used during the state transfer process to dynamically decide whether the current event stream can be aggregated into at least one composite teaching event, perform the aggregation step, and determine the event type of each composite teaching event.

[0104] The following is an example to illustrate. Figure 6 , Figure 6 The behavior event sequence of a certain classmate is shown in FIG. 2. Referring to Table 2, a transition condition of a classroom state is defined in Table 2. From the behavior event sequence of the classmate, it is determined that standing up to going on stage is a continuous on-stage interactive action. The correct data statistical result is to record the number of on-stage interactions of the classmate + 1. However, the traditional method will split standing up and going on stage, resulting in the number of standing up to speak + 1, and the number of on-stage interactions + 1. However, in fact, standing up is for on-stage interaction, not standing up to speak, which leads to a misjudgment of interactive behavior. In this embodiment, a state machine is used to perform state transition according to the event flow, and whether it is a continuous action is determined according to the state transition process. Referring to FIG. Figure 6 and Table 2, Figure 6 The behavioral event stream shown is input into the state machine. When the last event is input, the state machine is in the S2-student on-stage interaction state. The event stream is aggregated into a compound teaching event, and the corresponding event type is determined to be the on-stage interaction time. It is reversely inferred that the intention of the standing up behavior in the compound teaching event is to interact on the stage, rather than to stand up and answer questions. When identifying interactive behaviors, only the number of on-stage interactions is counted as +1, and both the standing up to speak and the on-stage interaction are not counted as +1, thereby improving the accuracy of effective interactive behavior statistics.

[0105] Table 2:

[0106] / A1-Stand Up A2-Students interact on stage S0-class start status S1 / S1-Student standing up S1 S2 S2-Students on stage interacting / S2

[0107] In this embodiment, the behavior events in the behavior event stream are obtained in sequence; when the obtained behavior events trigger the transfer conditions of the current classroom state, the classroom state transfer operation is performed; based on the dynamically transferred classroom state, the behavior event stream is aggregated into at least one compound teaching event, and the event type of each compound teaching event is determined. It is possible to subdivide various types of compound teaching events through classroom state transfer, reduce the code maintenance cost, sort out the effective interactive behaviors under the compound teaching events, improve the recognition accuracy of classroom interactive behaviors, and reduce the misjudgment rate of interactive behaviors.

[0108] In one embodiment, the process of determining whether an acquired behavior event triggers a transition condition of a current classroom state includes: obtaining a pre-set classroom state transition table, wherein the state transition table includes state transition conditions between classroom states; matching the acquired behavior event with each state transition condition corresponding to the current classroom state in the state transition table to obtain a matching result; and determining whether the acquired behavior event triggers a transition condition of the current classroom state based on the matching result.

[0109] The following definitions are pre-defined: define a non-empty finite classroom state set Q (state), define a non-empty finite behavior set A (event), and define the transition function (condition) between each classroom state. Specifically, set a classroom state transition table to manage various classroom states and the transition conditions between different classroom states. Figure 7 , Figure 7 A classroom state transition table is shown. In a specific implementation, a more detailed or more scenario classroom state transition table can be set according to actual needs, and this embodiment does not limit this. The limited classroom state set Q includes initialization, raising hands, standing up, asking questions, question and answer, discussion, going on stage, practice, reading together, and class end. The limited behavior set A includes hand-raising behavior events, standing up behavior events, group discussion events, going on stage behavior events, classroom practice events, teacher questioning events, reading together behavior events, non-speaker speech events, etc.

[0110] according to Figure 7 It can be seen that the X-axis represents the type of behavioral event, and the Y-axis represents the type of classroom state. (X, Y) means that in the current classroom state (i.e., the current state), when the behavioral event on the X-axis occurs (i.e., the state transition condition is met), the next state type (i.e., the next state) that needs to be migrated, and / or the action that needs to be performed. For example, Figure 7 When the current state is S4-questioning state, if a student stands up event occurs, check whether there is an E8 event (non-speaker speaking event) of the student within 20 seconds before and after (preset time window). If so, determine that the next state type to be migrated is the question-and-answer state, and the actions to be performed are: record the number of speeches + 1, check whether there is a hand-raising event within 20 seconds before and after, if so, record the number of hand-raising events + 1. Figure 7 The splitting rule means that when the teacher's question event E6 occurs, if the time interval between it and the previous event is greater than the set value, segment splitting is performed. If the teacher's question event E6 occurs, if the time interval between it and the previous event is less than or equal to the set value, it indicates that it is still in a continuous action link (such as continuous questioning and answering), and no splitting is required. The set value can be 20 seconds, or other time values ​​set according to actual needs, and this embodiment does not limit this.

[0111] Among them, when performing state transfer, the pre-set classroom state transfer table is queried according to the current classroom state (i.e., current state) and the obtained behavioral events (i.e., conditions) to determine the actions to be performed and / or the next state type to be migrated (i.e., next state).

[0112] In some implementations, a classroom state transition table is pre-set, the classroom state transition table is input into a finite state automaton, the state machine is initialized, the behavior event streams are input into the initialized state machine one by one, the state transition of the state machine is triggered, and the initialized state machine follows the following steps: Figure 7 Refer to the state transition logic in the classroom state transition table shown in Figure 8 , Figure 8 A process diagram of classroom state transfer is shown in , where S1~SN are various classroom states, A1~AN are various behavioral events in the event flow, and the state flow record includes classroom states that are dynamically transferred over time. Under the triggering of various behavioral events in the event flow, the classroom state is transferred to form a state flow record. In actual classrooms, since there are many types of classroom states and behavioral events, the transfer rules between different classroom states are complex and changeable. By setting the state transfer table of the state machine, various classroom states and transfer equations between states can be managed in a refined manner, so that the rules of compound teaching events can be quickly sorted out through the state machine. When adding or adjusting the logic in the future, it can still be achieved by configuring the state transfer table, which reduces the maintenance cost of the code.

[0113] In one embodiment, Fig. 9 As shown, step 506 includes:

[0114] Step 902, compare the dynamically transferred classroom status with the status conditions of each preset event type.

[0115] Among them, the preset event type refers to a plurality of predefined event types, such as question-and-answer scenario types, on-stage interactive scenario types, and the like. The state conditions of each preset event type are defined in advance. Optionally, the state conditions of each preset event type include at least one of an entry condition, a continuation condition, and an end condition. During the state transfer process, it is detected whether the classroom state at each moment continues to meet the state conditions of each preset event type. For example, after the state transfer, classroom state A migrates to classroom state B. If it is determined that classroom state A meets the state conditions of event type 1, it is further determined whether classroom state B meets the state conditions of event type 1.

[0116] In some implementations, for each preset event type, if the dynamically transferred classroom state at a certain moment meets the entry condition of the current preset event type, then at the next moment, it will continue to be judged whether the dynamically transferred classroom state meets the continuation condition of the current preset event type, until it is detected at a certain moment that the dynamically transferred classroom state meets the end condition of the current preset event type, and a composite teaching event is aggregated based on multiple classroom states in this continuous process.

[0117] The following is an example to illustrate:

[0118] The entry condition of the Q&A scene is defined as the question state, the continuation condition is the Q&A state, and the end condition is the question state. The teacher initiates the question (meets the entry condition), and the student raises his hand after the question is ready (effective interaction). The student stands up (effective interaction) or the student's voice is detected (meets the continuation condition), the student sits down, and the teacher initiates the next question (meets the end condition). In this way, the accuracy of hand-raising and standing interaction recognition is improved based on Q&A scene recognition.

[0119] The entry condition of the on-stage interaction scene is defined as the questioning state, the continuation condition is the on-stage interaction state, and the end condition is the student leaving the stage. The teacher initiates the question (meets the entry condition), the student stands up after the question is ready (effective interaction), the student stands up (not judged yet), the student is detected to be on stage (meets the continuation condition), and the student leaves the stage (meets the end condition). In this way, the accuracy of interaction recognition of raising hands, standing up, and on-stage interaction behaviors is improved based on on-stage interaction scene recognition.

[0120] Step 904: When the dynamically transferred classroom state meets the state condition of the target event type, determine the duration of the classroom state that meets the state condition of the target event type.

[0121] Among them, each dynamically transferred classroom state corresponds to a classroom time period. During the state transfer process, multiple classroom states meet the state conditions of the target event type. The duration period is determined according to the classroom time periods corresponding to the multiple classroom states that meet the state conditions of the target event type.

[0122] Step 906: Obtain a segment event stream from the behavior event stream according to the duration period.

[0123] The behavior event stream is sorted according to the class time, aligned according to the class time, and the segment event stream within the duration period is obtained from the behavior event stream.

[0124] Step 908 , aggregating the acquired fragment event streams into a composite teaching event, and determining the event type of the composite teaching event as the target event type.

[0125] Among them, based on the state transfer process, at least one fragment event stream is cut out from the behavior event stream, each fragment event stream corresponds to a composite teaching event, and the event type that meets the teaching event is determined according to the satisfied state conditions.

[0126] In this embodiment, the dynamically transferred classroom state is compared with the state conditions of each preset event type; when the dynamically transferred classroom state meets the state conditions of the target event type, the duration of the classroom state that meets the state conditions of the target event type is determined; based on the duration, a fragment event stream is obtained from the behavior event stream; the obtained fragment event stream is aggregated into a composite teaching event, and the event type of the composite teaching event is determined to be the target event type. In this way, various event types in the classroom can be subdivided according to the state conditions, which is convenient for sorting out various behaviors that may occur under various event types. By dividing the fragment event stream to analyze continuous actions, users can filter out invalid behavior events from the behavior event stream, screen out effective interactive behavior events, and improve recognition accuracy.

[0127] In one embodiment, step 208 includes: for each compound teaching event, determining at least one behavior event type according to the recognition strategy corresponding to the current compound teaching event; searching according to the at least one behavior event type from the fragment event stream corresponding to the current compound teaching event; and treating the searched behavior events as behavior events that match the recognition strategy.

[0128] Among them, each type of compound teaching event is set with a corresponding recognition strategy, and each recognition strategy corresponds to at least one behavior event type, for example, the question-and-answer scene event recognizes the number of hand-raising and speaking times, and the on-stage interaction scene recognizes the number of on-stage interactions, etc. According to the compound teaching event, the corresponding segment event stream is segmented from the behavior event stream, and the behavior event matching the behavior event type is searched from the segment event stream, and the effective interactive behaviors in the teaching classroom are recorded according to the searched behavior events.

[0129] In some implementations, the recognition strategy is provided with a search time window and a search starting point. The search starting point corresponds to a behavior event. The search time window can be 20 seconds before and after. According to the search time window and the search starting point, a search is performed from the segment event stream corresponding to the current composite teaching event according to at least one behavior event type, specifically including: determining at least one search starting point in the segment event stream, and searching for a behavior event matching at least one behavior event type from the search starting point according to the search time window. For example, the search starting point of the question-and-answer scene event is a standing event, the search time window is 20 seconds before and after, and the behavior event types are hand-raising events and speech events. If there are 3 standing events in the segment event stream, a hand-raising event and a speech event are searched in the 20 seconds before and after each standing event. If a hand-raising event is found, the number of hand-raising times is +1, and if a speech event is found, the number of speeches is +1.

[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0131] Based on the same inventive concept, the embodiment of the present application also provides a teaching interaction behavior recognition device for implementing the teaching interaction behavior recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more teaching interaction behavior recognition device embodiments provided below can refer to the limitations of the teaching interaction behavior recognition method above, and will not be repeated here.

[0132] In one embodiment, Fig.10 As shown, a teaching interactive behavior recognition device is provided, comprising:

[0133] The acquisition module 10 is used to acquire the behavior event stream in the teaching classroom.

[0134] The aggregation analysis module 20 is used to perform aggregation analysis on the behavior event flow to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event.

[0135] The determination module 30 is used to determine the recording strategy corresponding to each composite teaching event according to the event type of each composite teaching event.

[0136] The identification module 40 is used to use the identification strategies corresponding to each composite teaching event to identify the behavior events matching the identification strategies in the behavior event stream.

[0137] The interactive behavior determination module 50 is used to determine the interactive behavior in the teaching class according to the identified behavior events.

[0138] In the above-mentioned teaching interactive behavior recognition device, the behavioral event stream in the teaching classroom is obtained; the behavioral event stream is aggregated and analyzed to obtain at least one compound teaching event in the teaching classroom and the event type of each compound teaching event; according to the event type of each compound teaching event, the recognition strategy corresponding to each compound teaching event is determined; the recognition strategy corresponding to each compound teaching event is used respectively to identify the behavioral events matching the recognition strategy in the behavioral event stream; the interactive behavior in the teaching classroom is determined according to the identified behavioral events, and the compound teaching events can be aggregated on the basis of the behavioral events, and the effective interactive behaviors under the compound teaching events can be sorted out, thereby improving the recognition accuracy of classroom interactive behaviors and reducing the misjudgment rate of interactive behaviors.

[0139] In one embodiment, the aggregation analysis module 20 is also used to sequentially obtain behavioral events in the behavioral event stream; when the acquired behavioral events trigger the transfer conditions of the current classroom state, perform a classroom state transfer operation; aggregate the behavioral event stream into at least one compound teaching event based on the dynamically transferred classroom state, and determine the event type of each compound teaching event.

[0140] In one embodiment, the aggregation analysis module 20 is also used to obtain a pre-set classroom state transition table, which includes state transition conditions between various classroom states; match the acquired behavioral events with the various state transition conditions corresponding to the current classroom state in the state transition table to obtain a matching result; and determine whether the acquired behavioral events trigger the transition conditions of the current classroom state based on the matching result.

[0141] In one embodiment, the aggregation analysis module 20 is also used to compare the dynamically transferred classroom state with the state conditions of each preset event type respectively; when the dynamically transferred classroom state meets the state conditions of the target event type, determine the duration period of the classroom state that meets the state conditions of the target event type; according to the duration period, obtain a fragment event stream from the behavior event stream; aggregate the obtained fragment event stream into a compound teaching event, and determine that the event type of the compound teaching event is the target event type.

[0142] In one embodiment, the identification module 40 is also used to determine at least one behavior event type for each compound teaching event according to the identification strategy corresponding to the current compound teaching event; search according to the at least one behavior event type from the fragment event stream corresponding to the current compound teaching event; and use the searched behavior event as a behavior event that matches the identification strategy.

[0143] In one embodiment, the acquisition module 10 is further used to acquire video data and audio data in the teaching classroom; analyze the video data and the audio data to determine the behavior event flow in the teaching classroom.

[0144] In one embodiment, the teaching interaction behavior recognition device further includes a display module;

[0145] The display module is used to determine at least one teaching classroom corresponding to the interactive behavior display instruction when receiving the interactive behavior display instruction; search for the corresponding interactive behavior from the identification record file according to the at least one teaching classroom; classify and count the interactive behaviors according to the behavior types to obtain statistical results; and display the statistical results.

[0146] Each module in the above teaching interactive behavior recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0147] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store statistical data on interactive behaviors in teaching classrooms. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a teaching interactive behavior recognition method is implemented.

[0148] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0150] Obtain the behavioral event stream in the teaching classroom;

[0151] Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event;

[0152] According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event;

[0153] Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies;

[0154] The interactive behavior in the teaching class is determined based on the identified behavioral events.

[0155] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0156] Sequentially acquiring behavior events in the behavior event stream;

[0157] When the acquired behavior event triggers the transition condition of the current classroom state, the classroom state transition operation is executed;

[0158] The behavior event stream is aggregated into at least one composite teaching event based on the dynamically shifted classroom state, and the event type of each composite teaching event is determined.

[0159] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0160] Obtaining a preset classroom state transition table, wherein the state transition table includes state transition conditions between various classroom states;

[0161] Matching the acquired behavior events with each state transition condition corresponding to the current classroom state in the state transition table to obtain a matching result;

[0162] It is determined based on the matching result whether the acquired behavior event triggers the transfer condition of the current classroom status.

[0163] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0164] Compare the dynamically transferred classroom status with the status conditions of each preset event type respectively;

[0165] When the dynamically transferred classroom state meets the state condition of the target event type, determining the duration of the classroom state that meets the state condition of the target event type;

[0166] According to the duration period, obtaining a segment event stream from the behavior event stream;

[0167] The acquired fragment event streams are aggregated into a composite teaching event, and the event type of the composite teaching event is determined to be the target event type.

[0168] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0169] For each compound teaching event, determining at least one behavior event type according to the identification strategy corresponding to the current compound teaching event;

[0170] Searching according to the at least one behavior event type from the segment event stream corresponding to the current composite teaching event;

[0171] The searched behavior events are regarded as behavior events matching the identification strategy.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] Obtain video data and audio data in teaching classroom;

[0174] The video data and the audio data are analyzed to determine the flow of behavioral events in the teaching classroom.

[0175] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0176] Upon receiving the interactive behavior display instruction, determining at least one teaching classroom corresponding to the interactive behavior display instruction;

[0177] Searching for corresponding interactive behaviors from the identification record file according to the at least one teaching classroom;

[0178] Classifying and counting the interactive behaviors according to the types of behaviors to obtain statistical results;

[0179] The statistical results are displayed.

[0180] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0181] Obtain the behavioral event stream in the teaching classroom;

[0182] Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event;

[0183] According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event;

[0184] Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies;

[0185] The interactive behavior in the teaching class is determined based on the identified behavioral events.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] Sequentially acquiring behavior events in the behavior event stream;

[0188] When the acquired behavior event triggers the transition condition of the current classroom state, the classroom state transition operation is executed;

[0189] The behavior event stream is aggregated into at least one composite teaching event based on the dynamically shifted classroom state, and the event type of each composite teaching event is determined.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] Obtaining a preset classroom state transition table, wherein the state transition table includes state transition conditions between various classroom states;

[0192] Matching the acquired behavior events with each state transition condition corresponding to the current classroom state in the state transition table to obtain a matching result;

[0193] It is determined based on the matching result whether the acquired behavior event triggers the transfer condition of the current classroom status.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] Compare the dynamically transferred classroom status with the status conditions of each preset event type respectively;

[0196] When the dynamically transferred classroom state meets the state condition of the target event type, determining the duration of the classroom state that meets the state condition of the target event type;

[0197] According to the duration period, obtaining a segment event stream from the behavior event stream;

[0198] The acquired fragment event streams are aggregated into a composite teaching event, and the event type of the composite teaching event is determined to be the target event type.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0200] For each compound teaching event, determining at least one behavior event type according to the identification strategy corresponding to the current compound teaching event;

[0201] Searching according to the at least one behavior event type from the segment event stream corresponding to the current composite teaching event;

[0202] The searched behavior events are regarded as behavior events matching the identification strategy.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] Obtain video data and audio data in teaching classroom;

[0205] The video data and the audio data are analyzed to determine the flow of behavioral events in the teaching classroom.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] Upon receiving the interactive behavior display instruction, determining at least one teaching classroom corresponding to the interactive behavior display instruction;

[0208] Searching for corresponding interactive behaviors from the identification record file according to the at least one teaching classroom;

[0209] Classifying and counting the interactive behaviors according to the types of behaviors to obtain statistical results;

[0210] The statistical results are displayed.

[0211] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0212] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0213] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A teaching interactive behavior recognition method, characterized in that: The method comprises: Obtain the behavioral event stream in the teaching classroom; Performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and event types of each composite teaching event; According to the event type of each compound teaching event, determining the identification strategy corresponding to each compound teaching event; Using the identification strategies corresponding to the respective compound teaching events respectively, identifying the behavior events in the behavior event stream that match the identification strategies; The interactive behavior in the teaching class is determined based on the identified behavioral events.

2. The method according to claim 1, characterized in that The step of performing aggregate analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event includes: Sequentially acquiring behavior events in the behavior event stream; When the acquired behavior event triggers the transition condition of the current classroom state, the classroom state transition operation is executed; The behavior event stream is aggregated into at least one composite teaching event based on the dynamically shifted classroom state, and the event type of each composite teaching event is determined.

3. The method according to claim 2, characterized in that The process of determining whether the acquired behavior event triggers the transition condition of the current classroom state includes: Obtaining a preset classroom state transition table, wherein the state transition table includes state transition conditions between various classroom states; Matching the acquired behavior events with each state transition condition corresponding to the current classroom state in the state transition table to obtain a matching result; It is determined based on the matching result whether the acquired behavior event triggers the transfer condition of the current classroom status.

4. The method according to claim 2, characterized in that: The classroom state based on dynamic transfer aggregates the behavior event stream into at least one composite teaching event, and determines the event type of each composite teaching event, including: Compare the dynamically transferred classroom status with the status conditions of each preset event type respectively; When the dynamically transferred classroom state meets the state condition of the target event type, determining the duration of the classroom state that meets the state condition of the target event type; According to the duration period, obtaining a segment event stream from the behavior event stream; The acquired fragment event streams are aggregated into a composite teaching event, and the event type of the composite teaching event is determined to be the target event type.

5. The method according to claim 4, characterized in that The method of using the identification strategies corresponding to the respective compound teaching events to identify the behavior events in the behavior event stream that match the identification strategies includes: For each compound teaching event, determining at least one behavior event type according to the identification strategy corresponding to the current compound teaching event; Searching according to the at least one behavior event type from the segment event stream corresponding to the current composite teaching event; The searched behavior events are regarded as behavior events matching the identification strategy.

6. The method according to any one of claims 1 to 5, characterized in that: The method of obtaining the behavior event stream in the teaching classroom includes: Obtain video data and audio data in teaching classroom; The video data and the audio data are analyzed to determine the flow of behavioral events in the teaching classroom.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Upon receiving the interactive behavior display instruction, determining at least one teaching classroom corresponding to the interactive behavior display instruction; Searching for corresponding interactive behaviors from the identification record file according to the at least one teaching classroom; Classifying and counting the interactive behaviors according to the types of behaviors to obtain statistical results; The statistical results are displayed.

8. A teaching interactive behavior recognition device, characterized in that: The device comprises: The acquisition module is used to obtain the behavior event stream in the teaching classroom; An aggregation analysis module, used for performing aggregation analysis on the behavior event stream to obtain at least one composite teaching event in the teaching classroom and the event type of each composite teaching event; A determination module, used to determine the recording strategy corresponding to each composite teaching event according to the event type of each composite teaching event; An identification module, used to use the identification strategies corresponding to each composite teaching event respectively to identify the behavior events matching the identification strategies in the behavior event stream; The interactive behavior determination module is used to determine the interactive behavior in the teaching classroom according to the identified behavioral events.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images