Interactive control method, device and computer-readable storage medium for class inspection

Abstract

The present application discloses a class tour interactive control method, device and computer-readable storage medium, which relate to the field of network communication technology. The method comprises: based on the class tour authority associated with the class tour terminal, obtaining preview stream audio and video data corresponding to the classroom end; generating a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sending the combined preview screen to the class tour terminal; selecting at least one target classroom based on the selection operation of the class tour terminal for the combined preview screen; establishing a communication session between the class tour terminal and the classroom end corresponding to the target classroom to perform class tour interaction, overcoming the technical defects of the class tour system that interactive communication is impossible and only one-to-one interactive communication can be performed, and achieving the technical effect of improving the immediacy of communication in the class tour interactive system.

Description

Technical Field

[0001] The present application relates to the field of network communication technology, and in particular to a class tour interactive control method, device and computer-readable storage medium. Background Art

[0002] The classroom inspection system came into being as an innovative teaching management tool. By remotely monitoring the teaching situation in the classroom, the teaching status of the teacher and the learning situation of the students can be understood without the need for teaching management personnel to inspect each classroom.

[0003] With the current classroom inspection method, the inspector can observe the classroom situation unilaterally, but cannot provide timely feedback to the teaching teacher. The problems discovered and suggestions made by the inspector can often only be conveyed to the teaching teacher through other channels after class, and the inspector cannot interact with the classroom in real time.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a class tour interactive control method, device and computer-readable storage medium, aiming to solve the technical problem of how to improve the immediacy of communication in the class tour interactive system.

[0006] To achieve the above purpose, the present application proposes a class tour interactive control method, the method comprising:

[0007] Based on the patrol permissions associated with the patrol terminal, obtain the corresponding preview stream audio and video data of the classroom end;

[0008] Generate a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and send the combined preview screen to the class inspection terminal;

[0009] Based on the selection operation of the class inspection terminal on the combined preview screen, selecting at least one target classroom;

[0010] A communication session is established between the class inspection terminal and the classroom end corresponding to the target classroom to perform class inspection interaction.

[0011] In one embodiment, before the step of obtaining the preview stream audio and video data corresponding to the classroom end based on the class inspection authority associated with the class inspection terminal, the following steps are included:

[0012] According to the identity information corresponding to the class inspection terminal, the class inspection authority of the classroom terminal corresponding to the identity information is allocated.

[0013] In one embodiment, the step of generating a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sending the combined preview screen to the class inspection terminal includes:

[0014] Decoding the preview stream audio and video data;

[0015] The combined preview screen is generated according to the decoded preview stream audio and video data and sent to the class inspection terminal.

[0016] In one embodiment, the step of generating the combined preview screen according to the decoded preview stream audio and video data and sending the combined preview screen to the class inspection terminal includes:

[0017] Combining the preview stream audio and video data into a picture according to a preset layout to generate the combined preview picture;

[0018] Adjusting the playback speed of the preview stream audio and video data to ensure that the combined preview screen is displayed synchronously;

[0019] The combined preview picture is encoded and sent to the class inspection terminal.

[0020] In one embodiment, before the step of establishing a communication session between the class inspection terminal and the classroom end corresponding to the target classroom for class inspection interaction, the step includes:

[0021] Real-time acquisition of mainstream audio and video data of the class inspection terminal and the classroom terminal;

[0022] The mainstream audio and video data of the class inspection terminal and the classroom terminal are encoded and compressed.

[0023] In one embodiment, the step of establishing a communication session between the class inspection terminal and the classroom end corresponding to the target classroom to perform class inspection interaction includes:

[0024] The mainstream audio and video data of the class inspection terminal and the classroom end are transmitted to the other end point in real time through the network protocol, and a multi-party audio and video call session is established between the class inspection terminal and the classroom end;

[0025] The mainstream audio and video data are decoded to generate a combined picture corresponding to the class inspection terminal and the classroom terminal.

[0026] In one embodiment, the step of decoding the mainstream audio and video data to generate a combined picture corresponding to the class inspection terminal and the classroom terminal includes:

[0027] Decoding the mainstream audio and video data, generating and displaying a combined screen of the class inspection terminal according to a preset layout of the class inspection terminal;

[0028] The mainstream audio and video data are decoded, and a combined picture of the classroom end is generated and displayed according to the preset layout of the classroom end.

[0029] In one embodiment, after the step of establishing a communication session between the class inspection terminal and the classroom end corresponding to the target classroom for class inspection interaction, the following steps are included:

[0030] Receiving control instructions from the class inspection terminal and instructions for changing the target classroom;

[0031] According to the interactive state of the classroom end and the instruction, the combined screen corresponding to the class inspection terminal and the classroom end is adjusted.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a class tour interactive control device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the class tour interactive control method as described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the interactive control method of class tour described above are implemented.

[0034] The present application provides a class patrol interactive control method. The present application firstly obtains the audio and video preview stream of the classroom in real time through the real-time audio and video data transmission between the class patrol terminal and the classroom end, thereby improving the efficiency of the class patrol. By generating and sending a combined preview screen to the class patrol terminal, the real-time teaching situation of multiple classrooms can be viewed, and the classroom that needs to be interacted with can be selected according to the classroom situation, thereby improving the pertinence of the class patrol. By establishing a communication session between the class patrol terminal and the target classroom end, the class patrol teacher can directly interact and communicate with the teaching teacher, and immediately feedback the problems found or provide suggestions to the teaching teacher, thereby improving the immediacy of communication in the class patrol interactive system.

[0035] In summary, the present application obtains preview stream audio and video data corresponding to the classroom end based on the class patrol authority associated with the class patrol terminal; generates a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sends the combined preview screen to the class patrol terminal; selects at least one target classroom based on the selection operation of the class patrol terminal for the combined preview screen; establishes a communication session between the class patrol terminal and the classroom end corresponding to the target classroom to conduct class patrol interaction, thereby overcoming the technical defects of the class patrol system that interactive communication is impossible and can only be carried out on a one-to-one basis, and achieving the technical effect of improving the immediacy of communication in the class patrol interactive system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 A flow chart of the first embodiment of the interactive control method for class tour provided in this application;

[0039] Figure 2 This is a schematic diagram of the first scenario involved in Example 1 of the interactive control method for class tour of this application;

[0040] Figure 3 This is a schematic diagram of the second scenario involved in the first embodiment of the interactive control method for class tour of this application;

[0041] Figure 4 This is a schematic diagram of the third scenario involved in the first embodiment of the interactive control method for class tour of this application;

[0042] Figure 5 This is a schematic diagram of the fourth scenario involved in the first embodiment of the interactive control method for class tour of this application;

[0043] Figure 6 A flow chart of the second embodiment of the interactive control method for class tour provided in this application;

[0044] Figure 7 A flowchart of the third embodiment of the interactive control method for class tour provided in this application;

[0045] Figure 8 A flowchart diagram of the fourth embodiment of the interactive control method for class tour provided in this application;

[0046] Fig. 9 This is a schematic diagram of the first scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0047] Fig.10 This is a schematic diagram of the second scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0048] Fig.11 This is a schematic diagram of the third scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0049] Fig.12This is a schematic diagram of the fourth scenario involved in the fourth embodiment of the class tour interactive control method of this application;

[0050] Fig.13 This is a schematic diagram of the fifth scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0051] Fig.14 This is a schematic diagram of the sixth scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0052] Fig.15 This is a schematic diagram of the seventh scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0053] Fig.16 This is a schematic diagram of the eighth scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0054] Fig.17 This is a schematic diagram of the ninth scene involved in the fourth embodiment of the class tour interactive control method of this application;

[0055] Fig.18 This is a schematic diagram of the tenth scene involved in the fourth embodiment of the interactive control method for class tour of this application;

[0056] Fig.19 This is a schematic diagram of the eleventh scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0057] Fig. 20 This is a schematic diagram of the twelfth scenario involved in the fourth embodiment of the interactive control method for class tour of this application;

[0058] Fig.21 This is a schematic diagram of the thirteenth scene involved in the fourth embodiment of the interactive control method for class tour of this application;

[0059] Fig. 22 This is a schematic diagram of the fourteenth scene involved in the fourth embodiment of the interactive control method for class tour of this application;

[0060] Fig.23 This is a schematic diagram of the fifteenth scene involved in the fourth embodiment of the interactive control method for class tour of this application;

[0061] Fig.24 A flowchart of Embodiment 5 of the interactive control method for class tour provided in this application;

[0062] Fig.25 Schematic diagram of the device structure of the hardware operating environment involved in the interactive control method for class inspection in the embodiment of the present application.

[0063] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0064] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0065] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0066] The main solutions of the embodiments of this application are:

[0067] With the current classroom inspection method, the inspector can observe the classroom situation unilaterally, but cannot provide timely feedback to the teaching teacher. The problems discovered and suggestions made by the inspector can often only be conveyed to the teaching teacher through other channels after class, and the inspector cannot interact with the classroom in real time.

[0068] The present application obtains preview stream audio and video data corresponding to the classroom end based on the class inspection authority associated with the class inspection terminal; generates a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sends the combined preview screen to the class inspection terminal; selects at least one target classroom based on the selection operation of the class inspection terminal for the combined preview screen; establishes a communication session between the class inspection terminal and the classroom end corresponding to the target classroom to conduct class inspection interaction, thereby overcoming the technical defects of the class inspection system that interactive communication is impossible and can only be carried out on a one-to-one basis, and achieving the technical effect of improving the immediacy of communication in the class inspection interactive system.

[0069] It should be noted that the execution subject of this embodiment can be a class tour interactive control device, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., and this embodiment does not specifically limit this. The following takes the class tour interactive control device as an example to illustrate this embodiment and the following embodiments.

[0070] Embodiment 1

[0071] Based on this, this application proposes a first embodiment of the class tour interactive control method, please refer to Figure 1 , the class tour interactive control method comprises:

[0072] Step S100, based on the class inspection authority associated with the class inspection terminal, obtain the preview stream audio and video data corresponding to the classroom end.

[0073] In this embodiment, the class inspection authority is assigned according to the teacher's role and responsibilities, which determines which classrooms the teacher has the right to inspect, and determines which classrooms the teacher can view the preview stream audio and video data. The preview stream audio and video data is collected in real time by the audio and video equipment of the class inspection terminal and encoded into a format suitable for network transmission.

[0074] As an optional implementation, the login information of the class inspection terminal is verified, the class inspection authority information corresponding to the login information is obtained, all classroom information in the class inspection authority information is obtained, and the preview stream audio and video encoded and transmitted by the classroom end is obtained.

[0075] Step S200: Generate a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and send the combined preview screen to the class inspection terminal.

[0076] In this embodiment, the combined preview screen combines the preview stream video screens of multiple classrooms into an integrated video screen according to a certain layout and sequence, so that the teacher can view the situations of multiple classrooms at the same time.

[0077] As an optional implementation, the received preview stream audio and video data is decoded to determine the preview screen, and the preview screens of all classrooms are combined and displayed on the class inspection terminal interface according to a preset layout.

[0078] For example, please refer to Figure 2 When teacher A logs in to the system, the corresponding inspection permission information is searched according to the login information, and the classrooms A and B that the teacher has the inspection permission to be determined. The preview stream audio and video of these classrooms are obtained, and the received preview stream audio and video data are decoded. The preview screens of the preview stream audio and video of all classrooms are tiled into a window and displayed to the teacher.

[0079] For example, please refer to Figure 3 When teacher B logs in to the system, the corresponding inspection permission information is searched according to the login information, and the classrooms A, B and C that the teacher has the inspection permission are determined. The preview streaming audio and video of these classrooms are obtained, and the preview screens of the preview streaming audio and video of all classrooms are tiled into a window and displayed to the teacher.

[0080] Optionally, step S200 includes:

[0081] Step S210, decoding the preview stream audio and video data.

[0082] In this embodiment, the decoding process is the process of restoring the compressed preview stream audio and video data to the original audio and video signals. The decoder decodes the received data according to the algorithm and format used in the encoding to restore the playable audio and video content.

[0083] As an optional implementation, the format information of the preview stream audio and video data is parsed, and according to the parsed encoding format, a decoding algorithm is selected to decode the audio and video data to convert the preview stream audio and video data into a playable format.

[0084] Step S220, generating the combined preview screen according to the decoded preview stream audio and video data, and sending it to the class inspection terminal.

[0085] Optionally, step S220 includes:

[0086] Step S221, combining the preview stream audio and video data according to a preset layout to generate the combined preview picture.

[0087] In this embodiment, the preset layout is one or more predefined screen arrangements, and the size and position of each preview screen can be adjusted according to user needs.

[0088] As an optional implementation, a preset layout is determined according to the number of classrooms, and the preview screens are arranged according to preset positions and sizes to generate a combined preview screen.

[0089] Exemplarily, the preview screens are arranged in a grid format in the window. If there is one classroom, the preview screens are displayed separately in the window; if there are two classrooms, the preview screens are displayed side by side in the window; if there are three classrooms, the preview screens are displayed in the window with two screens side by side, and the third screen is located on the top and has the same width as the first two.

[0090] As another optional implementation, a preset layout is provided to the class inspection terminal for selection, and the teacher sets the position and size of each preview screen by dragging, resizing, and other operations on the class inspection terminal. Preview screens of multiple classrooms are generated based on the layout and adjustments selected by the teacher.

[0091] Step S222, adjusting the playback speed of the preview stream audio and video data to ensure that the combined preview screen is displayed synchronously.

[0092] Step S223: encode the combined preview image and send it to the class inspection terminal.

[0093] Step S300: selecting at least one target classroom based on the selection operation of the class inspection terminal on the combined preview screen.

[0094] In this embodiment, the selection operation refers to the selection action performed by the patrol teacher on one of the combined preview screens through the patrol terminal, and the target classroom refers to the classroom selected by the patrol terminal user through the selection operation for further monitoring or management.

[0095] As an optional implementation, one or more classrooms are selected for inspection according to the inspection terminal, the mainstream audio and video data on the classroom side is decoded, and the mainstream video images of all classrooms are combined and displayed on the teacher side interface according to a preset layout.

[0096] For example, please refer to Figure 4 , according to the classrooms A and B selected by teacher A through the class inspection terminal software, the mainstream audio and video data of these classrooms are decoded and processed, and the mainstream audio and video data of classrooms A and classroom B are combined into two pictures for teacher A to watch.

[0097] For example, please refer to Figure 5 , according to the classrooms A, B and C selected by teacher B through the class inspection terminal software, the mainstream audio and video data of these classrooms are decoded and processed, and combined into a three-screen picture for teacher B to watch.

[0098] Step S400, establishing a communication session between the class inspection terminal and the classroom end corresponding to the target classroom to perform class inspection interaction.

[0099] In this embodiment, a multi-party audio and video call session refers to a session in which multiple participants can simultaneously communicate in audio and video in a network environment. This session allows participants to see and hear the audio and video content of other participants in real time, thereby achieving remote communication and collaboration. The multi-party audio and video call session includes the patrol teacher of the patrol terminal and the teachers and students in the patrolled classrooms.

[0100] As an optional implementation method, a suitable network protocol is selected according to network conditions and real-time requirements, a connection between the class inspection terminal and the classroom end is established through the network protocol, mainstream audio and video data is encoded and transmitted in real time, the received audio and video data is decoded and processed by a corresponding decoder to restore it to the original audio and video data, and the corresponding images of the class inspection terminal and the classroom end are combined and displayed according to a preset layout.

[0101] For example, an audio and video call is established with the codec device in the classroom through the H323 / SIP protocol, and the audio and video data collected by the camera and microphone and sent by the codec device in the classroom through the RTP (Real-time Transport Protocol) protocol are received; at the same time, the audio and video data collected by the patrol terminal are sent to the codec device in the classroom through the RTP protocol, and the codec device then displays the audio and video data through a monitor or TV, and combines it into relevant multi-screens based on the current interactive status of the classroom and the teacher's audio and video data and sends it to the patrol terminal, and the teacher can see the combined multi-screens through a PC browser player.

[0102] The present embodiment provides a class tour interactive control method. The present embodiment first obtains preview stream audio and video data corresponding to the classroom end based on the class tour authority associated with the class tour terminal; generates a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sends the combined preview screen to the class tour terminal; selects at least one target classroom based on the selection operation of the class tour terminal on the combined preview screen; establishes a communication session between the class tour terminal and the classroom end corresponding to the target classroom to perform class tour interactive communication, thereby overcoming the technical defects of being unable to perform interactive communication and only being able to perform one-to-one interactive communication in the class tour system, and achieving the technical effect of improving the immediacy of communication in the class tour interactive system.

[0103] Based on the first embodiment, the second embodiment of the present application proposes a class tour interactive control method, referring to Figure 6 , before step S100, including:

[0104] Step S500: allocating class inspection rights of the classroom terminal corresponding to the identity information according to the identity information corresponding to the class inspection terminal.

[0105] In this embodiment, the identity information is a set of data used to uniquely identify the class patrol teacher, and is used to determine the class patrol authority of the class patrol teacher who logs into the class patrol terminal.

[0106] As an optional implementation method, specific rules for allocating classroom inspection permissions are defined according to the actual situation and needs of the school, and specific classroom inspection permissions are allocated according to the positions in the teacher's identity information.

[0107] For example, a teacher position is a school-level leader and head of the teaching management department, who has the authority to inspect all classrooms and can inspect any classroom and course; the head of a college has the authority to inspect the teaching area of ​​the college; and ordinary teachers have the authority to inspect the classes they teach.

[0108] This embodiment provides a class inspection interactive control method. This embodiment firstly formulates clear authority allocation rules and strictly implements them, so as to ensure the fairness of authority allocation and avoid the subjectivity of authority allocation. According to preset rules, class inspection authorities are automatically allocated to class inspection teachers, thereby improving the efficiency of class inspection authority allocation.

[0109] Based on the first embodiment, the third embodiment of the present application proposes a class tour interactive control method, referring to Figure 7 , before step S400, including:

[0110] Step S600, obtaining the mainstream audio and video data of the class inspection terminal and the classroom terminal in real time.

[0111] Step S700, encoding and compressing the mainstream audio and video data of the class inspection terminal and the classroom terminal.

[0112] In this embodiment, encoding refers to converting the original audio and video signals into a format that is more suitable for storage, transmission or processing. Compression is the process of reducing the size of data to save storage space or improve transmission efficiency.

[0113] As an optional implementation, the collected audio and video signals are preprocessed, a suitable audio and video encoder is selected according to the audio and video signal transmission requirements, the preprocessed audio and video signals are input into the encoder, the original audio and video signals are converted into encoded data streams, and compression processing is performed to reduce the amount of data.

[0114] This embodiment provides a class inspection interactive control method. This embodiment first obtains audio and video data from the class inspection terminal and the classroom end in real time, and reduces the data volume through encoding and compression technology to improve transmission efficiency, thereby ensuring that the class inspection teacher and the teaching teacher can obtain each other's data in real time. The teaching teachers receive external attention and support from the class inspection teachers during the teaching process, their teaching highlights can be recognized in a timely manner, and teaching problems can receive timely guidance. The class inspection system can give full play to its role in promoting the professional growth of teachers.

[0115] Based on the first embodiment, the fourth embodiment of the present application proposes a class tour interactive control method, referring to Figure 8 , step S400 includes:

[0116] Step S410: The mainstream audio and video data of the class inspection terminal and the classroom end are transmitted to the other end in real time through the network protocol, and a multi-party audio and video call session is established between the class inspection terminal and the classroom end.

[0117] In this embodiment, network protocols are important protocols for multimedia communication, including TCP / IP (Transmission Control Protocol / Internet Protocol), H323 protocol, SIP (Session Initiation Protocol), RTP, RTCP (RTP Control Protocol), RTSP (Real Time Streaming Protocol), etc. A multi-party audio and video call session allows two or more users to communicate audio and video at the same time, and each participant can see and hear the audio and video content of other participants.

[0118] For example, a UDP (User Datagram Protocol) connection is established between the class inspection terminal and each selected classroom terminal. The class inspection terminal encodes and compresses the collected audio and video data and sends it to each classroom terminal via a UDP data packet. At the same time, each classroom terminal also sends its own audio and video data to the class inspection terminal and other classroom terminals in the form of a UDP data packet.

[0119] Exemplarily, the class inspection interactive system establishes an audio and video call with the codec device in the classroom through the H323 protocol. The codec device in the classroom sends the audio and video data collected by the camera and microphone to the class inspection interactive system through the RTP protocol. At the same time, the class inspection interactive system sends the audio and video data collected by the teacher's PC to the codec device in the classroom through the RTP protocol, and the codec device then displays the audio and video data through a monitor or TV.

[0120] Exemplarily, the class inspection interactive system establishes an audio and video call with the codec device in the classroom through the SIP protocol. The codec device in the classroom sends the audio and video data collected by the camera and microphone to the class inspection interactive system through the RTP protocol. At the same time, the class inspection interactive system sends the audio and video data collected by the teacher's PC to the codec device in the classroom through the RTP protocol, and the codec device then displays the audio and video data through a monitor or TV.

[0121] Step S420: decode the mainstream audio and video data to generate a combined picture corresponding to the class inspection terminal and the classroom terminal.

[0122] In this embodiment, the combined screen refers to the result of merging multiple audio and video sources into one interface according to a preset layout. The preset layout can be designed according to actual needs, such as a four-split screen (the teacher's screen occupies most of the area, and the remaining three areas display the screens of the three classrooms respectively), a three-split screen (the teacher's screen and two classroom screens are displayed side by side), a single screen + a small window (the teacher's screen is the main one, and one or more classroom screens are displayed in the form of small windows).

[0123] Exemplarily, the system supports a classroom being interacted with by multiple teachers at the same time. When only teacher A interacts, the screen in classroom A is the screen of teacher A, the sound is the sound of teacher A, the screen of teacher A's PC browser is the screen of classroom A and teacher A, and the sound is the sound of classroom A. When teacher A interacts with classroom B at the same time, the screen of classroom A is a multi-screen combination of teacher A and classroom B, the sound is the sound of teacher A and classroom B, the screen of teacher A's PC browser is a multi-screen combination of all the interactive classroom screens and teacher A's screen, and the sound is the sound of all the interactive classrooms. When teachers A and B interact with classroom A at the same time, the screen of classroom A is a multi-screen combination of the screens of teachers A and teacher B and the screens of other classrooms B that teachers A and teacher B interact with at the same time, and the sound is the sound of all teachers and the sound of classroom B that teachers A and teacher B interact with at the same time; the screens displayed by teachers A and teacher B's PC browsers are small by default, and double-clicking can enlarge the display to the same size as the screen of the classroom. During the interaction, the screen corresponding to the speaker has a flashing red frame as a prompt.

[0124] Optionally, step S420 includes:

[0125] Step S421, decoding the mainstream audio and video data, generating and displaying a combined screen of the class inspection terminal according to a preset layout of the class inspection terminal.

[0126] For example, when teacher A interacts with classroom A and classroom B at the same time, the students in classroom A speak. Fig. 9 The screen that teacher A sees on his PC is a combination of classroom A screen, classroom B screen, and teacher A's local screen. The classroom A sub-screen flashes a red frame to indicate that he is speaking. Teacher A double-clicks to enlarge his own screen. Please refer to Fig.10 What teacher A sees on his PC is a three-screen combination of classroom A screen, classroom B screen, and teacher A's local screen. The classroom A sub-screen flashes a red frame to indicate that someone is speaking.

[0127] For example, when teacher A interacts with classroom A and classroom B at the same time, and teacher B interacts with classroom A, the students in classroom A speak, please refer to Fig.11 , Teacher A's PC sees a combination of classroom A and teacher B, classroom B, and teacher A's local screen. The combination of classroom A and teacher B is a combination of classroom A and teacher B, with a large and a small screen. The classroom A sub-screen flashes a red frame to indicate that the speaker is speaking. Teacher A double-clicks to enlarge his own screen, please refer to Fig.12, Teacher A's PC sees a three-screen screen consisting of the combined screen of Classroom A and Teacher B, the screen of Classroom B, and the local screen of Teacher A. The sub-screen of Classroom A flashes a red frame to indicate that the teacher is speaking. At this time, Teacher B interacts with Classroom C again. Please refer to Fig.13 , Teacher B sees a screen composed of two large and one small three-screen screens, which is composed of classroom A and teacher A's screen, classroom C's screen, and teacher B's local screen. The red frame on classroom A flashes to indicate that a speech is being made. Teacher B double-clicks to enlarge his own screen, please refer to Fig.14 , Teacher B sees three screens divided into three parts: the combined screen of classroom A and teacher A, the screen of classroom C, and the local screen of teacher B. The sub-screen of classroom A flashes a red frame to indicate that the teacher is speaking.

[0128] For example, when teacher A interacts with classroom A and classroom B, teacher B interacts with classroom A and classroom C, teacher C interacts with classroom A, classroom B and classroom C, and teacher D interacts with classroom B, the students in classroom A speak at this time, please refer to Fig.15 , Teacher A's PC sees three screens: classroom A combined screen, classroom B combined screen, and teacher A's local screen. The classroom A combined screen is a combination of classroom A, teacher B, and teacher C screens into a large and two small screens. The classroom B combined screen is a combination of classroom B, teacher C, and teacher D screens into a large and two small screens. The classroom A sub-screen flashes a red frame to indicate that the speaker is speaking. Teacher A double-clicks to enlarge his own screen, please refer to Fig.16 , what teacher A sees on his PC is a three-screen display consisting of the classroom A combined screen, the classroom B combined screen, and the teacher A local screen. The classroom A sub-screen has a flashing red frame to indicate that the teacher is speaking.

[0129] Step S422, decoding the mainstream audio and video data, generating and displaying a combined screen of the classroom end according to the preset layout of the classroom end.

[0130] For example, when teacher A interacts with classroom A and classroom B at the same time, the students in classroom A speak. Fig.17 The decoded image displayed by the codec device in classroom A is a two-image screen composed of the teacher A image and classroom B image of the classroom tour interactive system. Please refer to Fig.18 The picture decoded and displayed by the encoding and decoding equipment in classroom B is a two-screen combination of the teacher A screen and the classroom A screen of the interactive system. The classroom A sub-screen flashes a red frame to indicate that someone is speaking.

[0131] For example, when teacher A interacts with classroom A and classroom B at the same time, and teacher B interacts with classroom A, the students in classroom A speak, please refer to Fig.19 The screen displayed in classroom A is a combination of teacher A screen, classroom B screen and teacher B screen divided into three parts; please refer to Fig.18The decoded screen displayed by the codec device in classroom B is a screen composed of teacher A and classroom A, and the sub-screen of classroom A flashes a red frame to indicate that the teacher is speaking. At this time, teacher B interacts with classroom C again. Please refer to Fig. 20 The screen of classroom A is composed of teacher A screen, teacher B screen, classroom B screen and classroom C screen. Please refer to Fig.21 The screen of classroom C is a screen divided into two parts by the screen of teacher C and the screen of classroom A. The red frame of the sub-screen of classroom A flashes to indicate that a speaker is speaking.

[0132] For example, when teacher A interacts with classroom A and classroom B, teacher B interacts with classroom A and classroom C, teacher C interacts with classroom A, classroom B and classroom C, and teacher D interacts with classroom B, the students in classroom A speak at this time, please refer to Fig. 22 The screen displayed in classroom B is a combination of teacher A screen, classroom A screen, teacher C screen, teacher D screen and classroom C screen. The red frame of classroom A sub-screen flashes to indicate that someone is speaking; please refer to Fig.23 The screen of classroom C is a four-screen combination of teacher B screen, teacher C screen, classroom A screen and classroom B screen.

[0133] As an example of this embodiment, every time a teacher adds an interactive classroom, the teacher's screen will have an additional screen of the interactive classroom, and the interactive classroom screen will also have an additional teacher screen of the teacher, and the interactive classroom screen will form an n-equally divided screen with the screens of other teachers who are interacting with the classroom. The sub-screen of the classroom corresponding to the screen seen by other teachers interacting with the classroom will have an additional small screen of the teacher, indicating that the classroom corresponding to this sub-screen is also being interacted by the teacher.

[0134] The present embodiment provides a class tour interactive control method. The present embodiment first implements real-time audio and video transmission and multi-party audio and video call sessions between the class tour terminal and the classroom end through a network protocol, thereby improving the efficiency of class tour interaction. The pictures of the class tour terminal and each classroom end are combined and displayed according to a preset layout, so that all participants can clearly see each other's pictures. The class tour teacher can interact with the students in the classroom, fully understand the students' learning status and needs, determine the students' feelings and feedback as the teaching subject, and flexibly adjust the layout according to teaching needs. The class tour teacher can communicate with the teaching teacher in real time, point out the highlights and shortcomings in the teaching, and help the teaching teacher to adjust the teaching strategy in time, optimize the teaching effect, and improve the quality of class tour interaction.

[0135] Based on the first embodiment, the fifth embodiment of the present application proposes a class tour interactive control method, referring to Fig.24 , after step S400, including:

[0136] Step S800, receiving the control instruction of the class inspection terminal and the target classroom change instruction.

[0137] In this embodiment, the control instruction is an instruction sent by the class inspection terminal to the inspector to adjust the screen display.

[0138] Step S900, adjusting the combined screen corresponding to the class inspection terminal and the classroom end according to the interactive state of the classroom end and the instruction.

[0139] In this embodiment, the interactive status is the status of the classroom being interacted by the patrol teachers, including the number of interactive patrol teachers.

[0140] As an optional implementation method, the interactive status of the classroom is changed according to the interaction of the patrol terminal with the classroom during the interaction process. If the teacher cancels the interaction with the classroom, the screen of the patrol terminal is adjusted to remove the classroom screen, and the screen is adjusted according to the instructions of the teacher of the patrol terminal through clicks, drags, etc.

[0141] For example, when teacher A interacts with classroom A and classroom B at the same time, please refer to Fig.17 The decoded screen displayed by the codec device in classroom A is a screen composed of teacher A and classroom B divided into two equal parts by the interactive system. At this time, teacher B interacts with classroom A and classroom C. Please refer to Fig. 20 The screen of classroom A becomes a four-screen composed of teacher A screen, teacher B screen, classroom B screen and classroom C screen.

[0142] For example, when teacher A interacts with classroom A and classroom B at the same time, the students in classroom A speak. Fig. 9 The screen that teacher A sees on his PC is a combination of classroom A screen, classroom B screen, and teacher A's local screen. The classroom A sub-screen flashes a red frame to indicate that he is speaking. Teacher A double-clicks to enlarge his own screen. Please refer to Fig.10 , what teacher A sees on his PC is the classroom A screen, classroom B screen, and teacher A's local screen, which are divided into three parts by the interactive system. The red frame of classroom A sub-screen flashes to indicate that the teacher is speaking. On this basis, when teacher B interacts with classroom A again, the students in classroom A speak at this time. Please refer to Fig.11 , Teacher A's PC sees a combination of classroom A and teacher B, classroom B, and teacher A's local screen. The combination of classroom A and teacher B is a combination of classroom A and teacher B, with a large and a small screen. The classroom A sub-screen flashes a red frame to indicate that the speaker is speaking. Teacher A double-clicks to enlarge his own screen, please refer to Fig.12, what teacher A sees on his PC is a three-screen display consisting of the combined screen of classroom A and teacher B, the screen of classroom B, and the local screen of teacher A. The sub-screen of classroom A flashes a red frame to indicate that the teacher is speaking.

[0143] This embodiment provides a class inspection interactive control method. This embodiment first dynamically adjusts the layout and arrangement of the screen according to the needs of the teacher so as to inspect multiple classrooms at the same time, thereby improving the efficiency of the class inspection and reducing the cost of the class inspection. The screen combination is adjusted in real time according to the interactive status of the classroom end and the control instructions of the teacher end. The inspecting teacher can collect more information about classroom teaching, including the self-reflection of the teaching teacher, direct feedback from students, etc., and the inspecting teacher can evaluate the effect of the class inspection more comprehensively and objectively.

[0144] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the interactive control method of class tours in the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0145] The present application provides a class tour interactive control device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the class tour interactive control method in the above-mentioned embodiment one.

[0146] Reference below Fig.25 , which shows a schematic diagram of the structure of a class tour interactive control device suitable for implementing the embodiment of the present application. The class tour interactive control device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.25 The interactive control device for class tour shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0147] like Fig.25As shown, the class tour interactive control device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the class tour interactive control device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the class tour interactive control device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a class tour interactive control device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0148] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0149] The interactive control device for class tour provided by the present application adopts the interactive control method for class tour in the above embodiment, which can solve the technical problem of how to improve the immediacy of communication in the interactive control system for class tour. Compared with the prior art, the beneficial effects of the interactive control device for class tour provided by the present application are the same as the beneficial effects of the interactive control method for class tour provided by the above embodiment, and other technical features in the interactive control device for class tour are the same as those disclosed in the method of the previous embodiment, which will not be described in detail here.

[0150] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0152] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the class tour interactive control method in the above-mentioned embodiment.

[0153] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0154] The above-mentioned computer-readable storage medium may be included in the interactive control device for class tour; or it may exist independently without being assembled into the interactive control device for class tour.

[0155] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the interactive control device for class tour, the interactive control device for class tour can be written in one or more programming languages ​​or a combination thereof to write a computer program code for performing the operation of the present application. The programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and a conventional procedural programming language, such as "C" language or a similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0156] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0158] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned interactive control method for class tours, and can solve the technical problem of how to improve the immediacy of communication in the interactive control system for class tours. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the interactive control method for class tours provided by the above-mentioned embodiments, and will not be elaborated here.

[0159] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A class tour interactive control method, characterized in that: The method includes: Based on the class inspection authority associated with the class inspection terminal, obtain the preview stream audio and video data corresponding to the classroom end, wherein the preview stream audio and video data is collected in real time by the audio and video equipment of the class inspection terminal and encoded into a format suitable for network transmission; Generate a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and send the combined preview screen to the class inspection terminal; Based on the selection operation of the class inspection terminal on the combined preview screen, selecting at least one target classroom; The mainstream audio and video data of the class patrol terminal and the classroom end are transmitted to the other end point in real time through the network protocol, and a multi-party audio and video call session is established between the class patrol terminal and the classroom end, wherein the multi-party audio and video call session allows more than two users to conduct audio and video communication at the same time, and each participant can see and hear the audio and video content of other participants; Decoding the mainstream audio and video data to generate a combined screen corresponding to the class inspection terminal and the classroom terminal, wherein the combined screen refers to the result of combining multiple audio and video sources into one interface according to a preset layout, and the preset layout is designed according to actual needs; Whenever a patrol teacher adds an interactive classroom, the screen of the patrol terminal corresponding to the patrol teacher will have an additional screen of the interactive classroom, and the screen of the classroom terminal corresponding to the interactive classroom will also have an additional screen of the patrol teacher, and the screen of the interactive classroom will be equally divided with the screens of other teachers who are interacting with the interactive classroom. The sub-screen of the interactive classroom corresponding to the screen seen by other teachers interacting with the interactive classroom will have an additional small screen of the patrol teacher, indicating that the interactive classroom corresponding to this sub-screen is also being interacted with by the patrol teacher.

2. The method according to claim 1, characterized in that The step of obtaining preview stream audio and video data corresponding to the classroom end based on the class inspection authority associated with the class inspection terminal, wherein the preview stream audio and video data is collected in real time by the audio and video equipment of the class inspection terminal and encoded into data in a format suitable for network transmission, includes: According to the identity information corresponding to the class inspection terminal, the class inspection authority of the classroom terminal corresponding to the identity information is allocated.

3. The method according to claim 1, characterized in that The step of generating a combined preview screen corresponding to the classroom end according to the preview stream audio and video data, and sending the combined preview screen to the class inspection terminal comprises: Decoding the preview stream audio and video data; The combined preview screen is generated according to the decoded preview stream audio and video data, and is sent to the class inspection terminal.

4. The method according to claim 3, characterized in that The step of generating the combined preview screen according to the decoded preview stream audio and video data and sending the combined preview screen to the class inspection terminal comprises: Combining the preview stream audio and video data into a picture according to a preset layout to generate the combined preview picture; Adjusting the playback speed of the preview stream audio and video data to ensure that the combined preview screen is displayed synchronously; The combined preview picture is encoded and sent to the class inspection terminal.

5. The method according to claim 1, characterized in that The step of transmitting the mainstream audio and video data of the class inspection terminal and the classroom end to the other end in real time through the network protocol, establishing a multi-party audio and video call session between the class inspection terminal and the classroom end, wherein the multi-party audio and video call session allows more than two users to conduct audio and video communication at the same time, and each participant can see and hear the audio and video content of other participants, includes: Real-time acquisition of mainstream audio and video data of the class inspection terminal and the classroom terminal; The mainstream audio and video data of the class inspection terminal and the classroom terminal are encoded and compressed.

6. The method according to claim 1, characterized in that The mainstream audio and video data is decoded to generate a combined screen corresponding to the class inspection terminal and the classroom terminal, wherein the combined screen refers to the result of merging multiple audio and video sources into one interface according to a preset layout, and the preset layout is designed according to actual needs, including: Decoding the mainstream audio and video data, generating and displaying a combined screen of the class inspection terminal according to a preset layout of the class inspection terminal; The mainstream audio and video data are decoded, and a combined picture of the classroom end is generated and displayed according to the preset layout of the classroom end.

7. The method according to claim 1, characterized in that The mainstream audio and video data is decoded to generate a combined screen corresponding to the class inspection terminal and the classroom terminal, wherein the combined screen refers to the result of merging multiple audio and video sources into one interface according to a preset layout, and the preset layout is designed according to actual needs, including: Receiving control instructions from the class inspection terminal and instructions for changing the target classroom; According to the interactive state of the classroom end, the control instruction and the change instruction, the combined screen corresponding to the class inspection terminal and the classroom end is adjusted.

8. A class tour interactive control device, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the class tour interactive control method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the class tour interactive control method as described in any one of claims 1 to 7 are implemented.

Images