A method for adjusting content coding in AI-powered online education

Abstract

The application discloses a content coding adjustment method for AI online education, which is applied between a server, a teacher end and all student ends, wherein the teacher end and each student end encodes captured video data and audio data to obtain original video streams of each end, and sends the original video streams to the server, and the server stores related parameters of the original video streams into a mapping table. The content coding adjustment method for the AI online education combines the multi-path original video streams of the student end, reduces pressure transmission, improves the efficiency of data transmission, reduces the network bandwidth demand, and meanwhile, the teacher end or the student end can request video streams with different resolutions and formats according to the network status and processing capacity of the teacher end or the student end, so that the fluency of the watching experience is ensured. Meanwhile, the number of video streams is reduced, the processing burden of the teacher end and the student end devices is reduced, and CPU and memory resources are saved.

Description

Technical Field

[0001] This invention belongs to the field of online education technology, specifically relating to a method for adjusting content encoding in AI-powered online education. Background Technology

[0002] In online education, specifically for remote learning scenarios, teachers and students use dedicated online education platforms for teaching activities. Teachers and students typically access a virtual classroom by logging into the platform using electronic devices such as computers or tablets equipped with cameras and microphones. A common video layout is that the teacher's video feed occupies the main position, while the student's video window is smaller and arranged in a pane-like pattern on one side or at the bottom of the screen. Teachers and students can communicate visually through the camera, seeing each other's facial expressions and body language; they can communicate in real-time through microphones and speakers; and students can type questions or teachers can post text messages in the chat window. Teachers can also use the platform for screen sharing, whiteboard interaction, and other functions.

[0003] From the student's perspective, the video feed on the student's end will include one teacher's video feed and multiple student video feeds (in a pane-by-pane format, e.g., 2). 2, 3 3. It also supports pane switching to display more student video feeds.

[0004] In existing technologies, to display video on the student's end, traditional solutions require receiving at least one teacher-side video stream and multiple student-side video streams. The size of the video pane determines the number of video streams the student needs to receive. For example, configuring one teacher-side video stream and one... The student-side window needs to receive 5 video streams (1 teacher + 4 students), while configuring 1 teacher-side video stream and 3... A 3-pane layout requires receiving 10 video streams (1 teacher + 9 students). While technologies like video compression (compressing the video stream before transmission using encoding techniques such as H.264 or H.265 to reduce file size without sacrificing quality) and adaptive streaming (automatically adjusting video quality based on student network conditions to ensure a smooth viewing experience; if a student's network bandwidth is limited, the video stream quality automatically decreases to maintain playback smoothness) reduce bandwidth requirements, overall, the student end still faces significant bandwidth and decoding pressure due to receiving multiple video streams. Each video stream requires a certain amount of bandwidth for transmission, and the student-side decoder needs to process these streams to display the video. Furthermore, as the pane size increases (e.g., 3...), the bandwidth requirements increase. 4,4 (Level 4), the problem will become even more prominent. Summary of the Invention

[0005] The purpose of this invention is to address the problems raised in the background art by proposing a method for adjusting the content encoding of AI online education.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention proposes a content encoding adjustment method for AI online education, which is applied between the server, the teacher's end, and all student ends, wherein:

[0008] The teacher's end and each student's end encode the captured video and audio data to obtain the original video stream of each end, and send the original video stream to the server. The server stores the relevant parameters of each original video stream in a mapping table.

[0009] The server will send the received raw video streams according to the number of raw video streams requested by the teacher or student. The teacher and student will play and display the received raw video streams in the first page of the preset area in the form of a preset pane, and each window in the pane will play and display one raw video stream from the student.

[0010] The teacher or student terminal monitors the frame rate of each original video stream in the current pane of the local playback in real time. When the frame rate is less than the first preset value, a request to merge video streams is sent to the server, requesting that the original video streams of multiple student terminals in the current pane of the student terminal be merged into one.

[0011] The server calculates the original video stream ID number of the student terminal associated with the request information based on the parameters of the request information, merges the original video streams of the student terminal with the obtained associated ID number to obtain the corresponding merged video stream, and stores the relevant parameters of the merged video stream in the mapping table.

[0012] The server then sends a request-response message for merging video streams to the teacher or student. If the operation result of the request-response message for merging video streams is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student.

[0013] After the teacher or student receives the merged video stream, it will be played and displayed in the current pane.

[0014] The teacher or student client can also switch the layout type, page number, and video stream resolution of the pane displaying the merged video stream.

[0015] Preferably, the ID sequence of all student-end original video streams is as follows: ;

[0016] For the original video stream of the student's device displayed in the pane on either the teacher's or student's end, if no student is speaking, the original video stream of the student's device is displayed sequentially in each window according to its ID number sequence. The order in which the original video stream of the student's device is displayed in all windows is called the video stream order, and is represented as follows: ,in This indicates the first window in all panes. The video stream corresponding to each window, and ;

[0017] When a student is speaking, the original video stream position of the student's terminal is swapped with the position of the window in the first row and column of the first page pane. The server swaps the positions of the two windows by regenerating the video stream of the pane where the two swapped window positions are located. Each time a position swap occurs, the server records the latest video stream order.

[0018] Preferably, when the teacher or student switches the layout type, page number, or video stream resolution of the pane displaying the merged video stream, they send a request for pane layout type switching, a request for pane page number switching, or a request for video stream resolution switching to the server, respectively.

[0019] The server looks up the corresponding merged video stream in the mapping table based on the pane layout type, pane page number, and video stream resolution in the request information for switching pane layout type, switching pane page number, or switching video stream resolution.

[0020] If the request is not found in the mapping table, the server calculates the ID number of the original video stream associated with the student based on the parameters of the request information for switching pane layout type, switching pane page number, or switching video stream resolution, as well as the latest video stream order. The server then performs the latest merging of the original video streams associated with the student with the obtained ID number to obtain the corresponding merged video stream and stores the relevant parameters of the merged video stream in the mapping table.

[0021] When the corresponding merged video stream is found or after the latest merge, the server sends a request response message to the teacher or student to switch pane layout type, switch pane page number, or switch video stream resolution. If the operation result of the request response message to switch pane layout type, switch pane page number, or switch video stream resolution is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student.

[0022] Preferably, the request information for merging video streams, switching pane layout types, switching pane page numbers, and switching video stream resolution all include parameters such as request information ID, requester ID, course instance ID, operation type, pane layout type, pane page number, and video stream resolution.

[0023] The request and response messages for merging video streams, switching pane layout types, switching pane page numbers, and switching video stream resolution all include a response message ID, a requester ID, a course instance ID, an operation result, a merged video stream ID, and a merged video address.

[0024] Preferably, the server calculates the ID number of the original student video stream associated with the request information based on parameters in the request information for merging video streams, the request information for switching pane layout types, the request information for switching pane page numbers, or the request information for switching video stream resolution. The server then merges the original student video streams with the obtained ID number to obtain the corresponding merged video stream and stores the relevant parameters of the merged video stream in a mapping table, including:

[0025] The server determines the pane layout type based on the request information for merging video streams, switching pane layout types, switching pane page numbers, or switching video stream resolutions. and pane page numbers And according to the latest video stream order The original video stream ID sequence associated with the student terminal of the request information was calculated as follows: The original video streams from the student terminals with the associated ID numbers are then merged into a single merged video stream.

[0026] The resolution of the obtained merged video stream is transcoded into the corresponding video resolution in the merged video stream request information, pane layout type switching request information, pane page number switching request information, or video stream resolution switching request information.

[0027] Generate a merged video stream ID and a merged video stream address from the transcoded merged video stream;

[0028] Establish a mapping relationship between the merged video stream and the corresponding pane layout type, pane page number, video stream resolution, merged video stream ID, merged video stream address, and the ID number of the original video stream associated with the student terminal, and store it in the mapping table. For each original video stream, the mapping table stores multiple video resolution levels, multiple original video stream IDs, and multiple original video stream addresses corresponding to each original video stream.

[0029] Preferably, for the teacher's end, the received original video stream or merged video stream from the student's end is displayed in the first area of ​​the playback interface in the form of a pane, and the received original video stream from the teacher's end is displayed in the second area of ​​the playback interface. The playback interface also has a third area for switching the pane layout type of the first area, a fourth area for switching the pane page number of the first area, and a fifth area for switching the resolution of each video stream in the first area.

[0030] For the student end, the received original video stream or merged video stream is displayed in the sixth area of ​​the playback interface in the form of a pane, and the received original video stream from the teacher end is displayed in the seventh area of ​​the playback interface. The playback interface also has an eighth area for switching the pane layout type of the sixth area, a ninth area for switching the pane page number of the sixth area, and a tenth area for switching the resolution of each video stream in the sixth area.

[0031] Preferably, the content encoding adjustment method for AI online education further includes enabling teachers or students to double-click the single-window video stream in the pane displaying the merged video stream to zoom in or out:

[0032] When double-clicking to zoom in, the teacher or student first calculates the number of rows and columns in the pane where the double-click event occurs, and the pane layout type is [not specified]. The coordinates of the top left corner of the pane are The coordinates of the bottom right corner of the pane are The position coordinates of the double-click event are The double-click event will then appear in the number of rows in the pane. The number of columns in the pane where the double-click event occurs. ,in Indicates rounding down;

[0033] Once the calculation is complete, the teacher or student sends a request to the server to zoom in on the single-window video stream in the pane. The request includes a request ID, a requester ID, a course instance ID, an operation type, a pane layout type, a pane page number, the zoomed-in video stream resolution, and the number of rows and columns in the pane where the double-click event occurs.

[0034] After receiving a request to zoom in on the video stream in a single window in the pane, the server searches the mapping table for the original video stream ID sequence of the associated student terminal corresponding to the merged video stream, based on the pane layout type, pane page number, and zoomed video stream resolution in the request message. Then, based on the row and column number of the double-click event in the pane and the pane layout type, the server obtains the ID number of the original video stream of the student terminal corresponding to the double-click event.

[0035] Then, look up the original video stream of the student with the corresponding ID number, the original video stream ID of the corresponding video stream resolution, and the original video stream address in the mapping table;

[0036] The server sends a request-response message to the teacher or student to zoom in on a single screen in the pane. The request-response message includes the response message ID, the requester ID, the course instance ID, the operation result, the original video stream ID corresponding to the zoomed-in single window, and the original video address.

[0037] If the operation result in the response message is successful, the server will send the original video stream of the student end with the corresponding ID number and video stream resolution to the requester with the corresponding ID.

[0038] After receiving the original video stream, the requester with the corresponding ID will play the original video stream over the entire first or sixth region.

[0039] Preferably, when the enlarged single-window video stream is zoomed out by double-clicking, the student or teacher sends a request to the server to zoom out by double-clicking the enlarged single-window video stream. The request information includes a request information ID, a requester ID, a course instance ID, an operation type, a pane layout type, a pane page number, and a video stream resolution.

[0040] After receiving a request to double-click to zoom out of the enlarged single-window video stream, the server looks up the corresponding merged video stream in the mapping table based on the pane layout type, pane page number, and video stream resolution in the request message.

[0041] The server sends a request response message to the teacher or student client requesting that the enlarged single-window video stream be zoomed out by double-clicking. This request response message includes a response message ID, requester ID, course instance ID, operation result, merged video stream ID, and merged video address. When the operation result is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student client.

[0042] Preferably, the content encoding adjustment method for AI online education further includes adaptive adjustment of video resolution:

[0043] When the size of the playback interface on the teacher's or student's end is smaller than the second preset value, the teacher's or student's end determines whether the resolution of each video stream displayed on the current playback interface is the preset minimum resolution. If the resolution of any video stream is not at the preset minimum, a request is sent to the server to request the corresponding video stream resolution to be adjusted. The video stream resolution in the request is the preset minimum.

[0044] When the playback interface on the teacher's or student's end is greater than or equal to the second preset value, the teacher's or student's end determines whether the resolution of each video stream displayed on the current playback interface is the preset highest resolution. If the resolution of any video stream is not at the preset highest resolution, a request message for adjusting the resolution of the corresponding video stream is sent to the server, and the resolution of the video stream in the request message is the preset highest resolution.

[0045] Preferably, when adaptively adjusting the video resolution, the method further includes determining whether the resolution of each video stream displayed on the current playback interface is at a preset minimum level when the teacher or student's end experiences playback stuttering under the combined video stream playback display. If the resolution of any video stream is not at the preset minimum level, the teacher or student's end sends a request to the server to switch the resolution of the corresponding video stream, and the video stream resolution in the request is the video resolution of the first preset level.

[0046] When adaptively adjusting the video resolution, the method also includes determining whether the resolution of each video stream displayed on the current playback interface is at the preset maximum level when the teacher or student is in the playback process of the merged video stream and the CPU utilization of the teacher or student is lower than the third preset value. If the resolution of any video stream is not at the preset maximum level, the teacher or student sends a request to the server to switch the resolution of the corresponding video stream, and the video stream resolution in the request is the video resolution of the second preset level.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] This AI-powered online education content encoding adjustment method merges multiple original video streams from the student's end, reducing transmission pressure, improving data transmission efficiency, and lowering network bandwidth requirements. Simultaneously, the teacher's or student's end can request video streams of different resolutions and formats based on their network conditions and processing capabilities, ensuring a smooth viewing experience. Furthermore, by reducing the number of video streams, the processing burden on both teacher's and student's devices is reduced, saving CPU and memory resources. Attached Figure Description

[0049] Figure 1 This is a display of the playback interface on the teacher's side of the present invention;

[0050] Figure 2 This is a display of the playback interface for the student side of this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] like Figures 1-2 As shown, a content encoding adjustment method for AI online education is provided, which is applied between the server, the teacher's end, and all student ends, wherein:

[0053] The teacher's end and each student's end encode the captured video and audio data (specifically, the teacher's end and each student's end capture video and audio data through their respective cameras and microphones and encode them), obtaining the original video streams of each end (i.e., obtaining the original video streams of the teacher's end and the original video streams of each student's end), and send the original video streams to the server (the teacher's end and each student's end send their respective encoded original video streams to the server), and the server stores the relevant parameters of each original video stream in a mapping table;

[0054] The server will send the received raw video streams according to the number of raw video streams requested by the teacher or student. The teacher and student will then display the received raw video streams in a preset pane format (e.g., 2) on the first page within a preset area. The video is displayed in two panes, and each window in the pane displays the original video stream from one student's end.

[0055] For the teacher's end, the received raw or merged video streams from the student's end are displayed in a pane-like format in the first area of ​​the playback interface (each pane occupies one page). The received raw video stream from the teacher's end is displayed in the second area of ​​the playback interface. The playback interface also includes a third area for switching the pane layout type in the first area, a fourth area for switching the pane page numbers in the first area, and a fifth area for switching the resolution of each video stream in the first area (e.g., ...). Figure 1 As shown, where Figure 1 In the diagram, the left pane is the first area, the raw video stream on the teacher's end in the lower right corner is the second area, switching panes corresponds to the third area, switching page numbers corresponds to the fourth area, and switching resolution corresponds to the fifth area.

[0056] For the student end, the received raw or merged video stream is displayed in panes (each pane occupies one page) in the sixth area of ​​the playback interface. The received raw video stream from the teacher end is displayed in the seventh area of ​​the playback interface. The playback interface also includes an eighth area for switching the pane layout type in the sixth area, a ninth area for switching the pane page numbers in the sixth area, and a tenth area for switching the resolution of each video stream in the sixth area (e.g., ...). Figure 2 As shown, where Figure 2 In the diagram, the right pane is the sixth area, the original video stream from the teacher's end on the left is the seventh area, switching panes corresponds to the eighth area, switching page numbers corresponds to the ninth area, and switching resolution corresponds to the tenth area.

[0057] The teacher or student client monitors the frame rate of each original video stream in the current pane of the local playback in real time. When the frame rate is lower than a first preset value (e.g., if the frame rate of one of the original video streams is detected to be lower than the first preset value), a request is sent to the server to merge the video streams, requesting that the original video streams from multiple student clients in the student client pane be merged into one (e.g., for 22 students, if calculated according to 2...). If the video is split into two panes, then there will be 6 merged video streams (2² = 4 × 5 + 2), meaning the first 5 merged video streams each contain 4 (2) 2) The 6th student's video feed contains the remaining 2 student's video feeds; if we follow the 3... If the video is split into three panes, then there will be three merged video streams (2² = 9 × 2 + 4), meaning the first two merged video streams each contain 9 (3 3) The third student's video feed contains the remaining 4 student's video feeds.

[0058] The server calculates the original video stream ID of the student associated with the request information based on the parameters of the request information (i.e., request information ID, requester ID (including the teacher's ID and all student's IDs), course instance ID, operation type, pane layout type, pane page number, and video stream resolution). It then merges the original video streams of the student associated with the obtained ID to obtain the corresponding merged video stream and stores the relevant parameters of the merged video stream (i.e., pane layout type, pane page number, video stream resolution, merged video stream ID, merged video stream address, and the ID of the original video stream associated with the student) in a mapping table.

[0059] The server then sends a request-response message for merging video streams to the teacher or student. If the operation result of the request-response message for merging video streams is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student.

[0060] After receiving the merged video stream, the teacher or student will play and display it in the current pane (the merged video stream is displayed in the pane in the following way: each window corresponds to the original video stream of the merged video stream before merging).

[0061] In one embodiment, the teacher's or student's end also switches the layout type, page number, and video stream resolution of the pane displaying the merged video stream.

[0062] Specifically, when the teacher or student switches the layout type, page number, or video stream resolution of the pane displaying the merged video stream, they send a request for pane layout type switching, a request for pane page number switching, or a request for video stream resolution switching to the server, respectively.

[0063] The server looks up the corresponding merged video stream in the mapping table based on the pane layout type, pane page number, and video stream resolution in the request information for switching pane layout type, switching pane page number, or switching video stream resolution.

[0064] If the request is not found in the mapping table, the server calculates the ID number of the original video stream associated with the student based on the parameters of the request information for switching pane layout type, switching pane page number, or switching video stream resolution, as well as the latest video stream order. The server then performs the latest merging of the original video streams associated with the student with the obtained ID number to obtain the corresponding merged video stream and stores the relevant parameters of the merged video stream in the mapping table.

[0065] When the corresponding merged video stream is found or after the latest merge, the server sends a request response message to the teacher or student to switch pane layout type, switch pane page number, or switch video stream resolution. If the operation result of the request response message to switch pane layout type, switch pane page number, or switch video stream resolution is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student.

[0066] It should be noted that the request information for merging video streams, switching pane layout types, switching pane page numbers, and switching video stream resolution all include the following parameters: request information ID, requester ID (including the teacher's ID and all student's IDs), course instance ID, operation type, pane layout type, pane page number, and video stream resolution. (Furthermore, the operation type in the request information for merging video streams is merging the original video stream from the student's end, and the operation type in the request information for switching pane layout types is switching pane layout types. In this embodiment, there are 1 pane layout type.) 1,2 2 and 3 In pane 3, the operation type in the request information for pane page number switching is pane page number switching, and the operation type in the request information for video stream resolution switching is video stream resolution switching. In this embodiment, the video stream resolution (including the original video stream from the teacher's end, the original video stream from the student's end, and the merged video stream) types are 480P, 720P, and 1080P.

[0067] The request and response messages for merging video streams, switching pane layout types, switching pane page numbers, and switching video stream resolution all include a response message ID, requester ID, course instance ID, operation result, merged video stream ID, and merged video address (where the operation result in the request and response message for merging video streams indicates whether the merged video stream was successful or failed, the operation result in the request and response message for switching pane layout types indicates whether the pane layout type was successfully or failed, the operation result in the request and response message for switching pane page numbers indicates whether the pane page number was successfully or failed, and the operation result in the request and response message for switching video stream resolution indicates whether the video stream resolution was successfully or failed).

[0068] In one embodiment, the ID sequence of all student-end raw video streams is: ;

[0069] For the original video stream of the student's device displayed in the pane on either the teacher's or student's end, if no student has spoken (and no student has yet spoken), the original video stream of the student's device is displayed sequentially in each window according to its ID number sequence. The order in which the original video stream of the student's device is displayed in all windows is called the video stream order, and is represented as follows: ,in This indicates the first window in all panes. The video stream corresponding to the nth window (i.e., after all panes and windows are arranged in order, the nth window) The video stream corresponding to the nth window, that is, regardless of the nth window Each window corresponds to a video stream from which student's original video stream is located in the ID sequence. To indicate the first (the video stream corresponding to each window), and (Among them, the panes are sorted according to their page numbers, and within a pane, first from left to right, then from top to bottom. For example, there are two pages, eight student windows, and the pane layout type is 2.) 2. The four windows in the pane with page number 1, from left to right and then from top to bottom, are the first window, the second window, the third window, and the fourth window, respectively. The four windows in the pane with page number 2, from left to right and then from top to bottom, are the fifth window, the sixth window, the seventh window, and the eighth window, respectively.

[0070] When a student is speaking, the original video stream position of the student's terminal is swapped with the position of the window in the first row and column of the first page pane. The server swaps the positions of the two windows by regenerating the video stream of the pane where the two swapped window positions are located. Each time a position swap occurs, the server records the latest video stream order.

[0071] In one embodiment, the server calculates the ID number of the original student video stream associated with the request information based on parameters in the request information for merging video streams, switching pane layout types, switching pane page numbers, or switching video stream resolution. The server then merges the original student video streams with the obtained ID number to obtain the corresponding merged video stream and stores the relevant parameters of the merged video stream in a mapping table, including:

[0072] The server determines the pane layout type based on the request information for merging video streams, switching pane layout types, switching pane page numbers, or switching video stream resolution. and pane page numbers And according to the latest video stream order The original video stream ID sequence associated with the student terminal of the request information was calculated as follows: (This sequence is obtained by sorting the windows in the pane from left to right and then from top to bottom), where, This indicates the first window in all panes. The video stream corresponding to each window This indicates the first window in all panes. The video streams corresponding to each window are merged into a single merged video stream by merging the original video streams from the student terminals with the associated ID numbers.

[0073] The resolution of the obtained merged video stream is transcoded into the corresponding video resolution in the merged video stream request information, pane layout type switching request information, pane page number switching request information, or video stream resolution switching request information.

[0074] Generate a merged video stream ID and a merged video stream address from the transcoded merged video stream;

[0075] Establish a mapping relationship between the merged video stream and its corresponding pane layout type, pane page number, video stream resolution, merged video stream ID, merged video stream address, and the ID number of the original video stream associated with the student client, and store this relationship in a mapping table. For each original video stream, the mapping table stores multiple video resolutions (480P, 720P, and 1080P), multiple original video stream IDs, and multiple original video stream addresses. The mapping table is shown in Table 1 (where N / A in Table 1 indicates an empty string).

[0076] Table 1

[0077]

[0078] In one embodiment, the content encoding adjustment method for AI online education further includes enabling teachers or students to double-click the single-window video stream in the pane displaying the merged video stream to zoom in or out.

[0079] When double-clicking to zoom in, the teacher or student first calculates the number of rows and columns in the pane where the double-click event occurs, and the pane layout type is [not specified]. The coordinates of the top left corner of the pane are The coordinates of the bottom right corner of the pane are The position coordinates of the double-click event are The double-click event will then appear in the number of rows in the pane. The number of columns in the pane where the double-click event occurs. ,in Indicates rounding down;

[0080] Once the calculation is complete, the teacher or student sends a request to the server to zoom in on the single-window video stream in the pane. This request includes the request ID, requester ID, course instance ID, operation type, pane layout type, pane page number, zoomed-in video stream resolution, and the row and column number of the double-click event within the pane (where the operation type in the request to zoom in on the single window in the pane is: zoom in on the single window in the pane).

[0081] After receiving a request to zoom in on the single-window video stream in the pane, the server, based on the pane layout type, pane page number, and video stream resolution in the request message, searches the mapping table for the original video stream ID sequence corresponding to the associated student's terminal for merging the video streams, and then determines the row and column number of the double-click event within the pane and the pane layout type. Using formula The double-click event is calculated to be the first in the original video stream ID sequence of the associated student terminal. The original video stream from each student's device is used to obtain the ID number of the original video stream from the student's device corresponding to the double-click event (e.g., the sequence of original video streams from the student's device with associated ID numbers is...). Based on the number of rows and columns the double-click event is in in the pane (second row, second column) and the pane layout type (2). 2. Using the formula The double-click event is calculated to be the 4th original video stream from the associated student's terminal in the original video stream ID sequence, thus the ID number of the original video stream from the student's terminal corresponding to the double-click event is 8).

[0082] Then, look up the original video stream ID and original video stream address of the student with the corresponding ID number in the mapping table;

[0083] The server sends a request-response message to the teacher or student to zoom in on a single screen in the pane. The request-response message includes the response message ID, the requester ID, the course instance ID, the operation result, the original video stream ID corresponding to the zoomed-in single window, and the original video address.

[0084] If the operation result in the response message is successful, the server will send the original video stream of the student end with the corresponding ID number and video stream resolution (i.e., the enlarged video stream resolution) to the requester with the corresponding ID.

[0085] After receiving the original video stream, the requester with the corresponding ID will play the original video stream in a way that covers the entire first or sixth region.

[0086] When double-clicking to zoom out of the enlarged single-window video stream, the student or teacher sends a request to the server to zoom out of the enlarged single-window video stream. This request includes a request ID, requester ID, course instance ID, operation type, pane layout type, pane page number, and video stream resolution (the operation type is to zoom out of the enlarged single-window video stream).

[0087] After receiving a request to double-click to zoom out of the enlarged single-window video stream, the server looks up the corresponding merged video stream in the mapping table based on the pane layout type, pane page number, and video stream resolution in the request message.

[0088] The server sends a request response message to the teacher or student client requesting that the enlarged single-window video stream be zoomed out by double-clicking. This request response message includes the response message ID, requester ID, course instance ID, operation result, merged video stream ID, and merged video address. When the operation result is successful, the server sends the merged video stream with the corresponding parameters to the teacher or student client (implementing the server to switch from one original video stream from a single student client to one virtual merged video stream before the original video stream is enlarged).

[0089] In one embodiment, the content encoding adjustment method for AI online education also includes adaptive adjustment of video resolution:

[0090] When the size of the playback interface on the teacher's or student's end is smaller than the second preset value, the teacher's or student's end determines whether the resolution of each video stream displayed on the current playback interface is the preset minimum resolution. If the resolution of any video stream is not at the preset minimum, a request is sent to the server to request the corresponding video stream resolution to be adjusted. The video stream resolution in the request is the preset minimum.

[0091] When the playback interface on the teacher's or student's end is greater than or equal to the second preset value, the teacher's or student's end determines whether the resolution of each video stream displayed on the current playback interface is the preset highest resolution. If the resolution of any video stream is not at the preset highest resolution, a request message for adjusting the resolution of the corresponding video stream is sent to the server, and the resolution of the video stream in the request message is the preset highest resolution.

[0092] When adaptively adjusting the video resolution, the system also includes a mechanism where, when the teacher or student is experiencing playback stuttering while playing a merged video stream (obtained by detecting the frame rate), the teacher or student determines whether the resolution of each video stream displayed on the current playback interface is at the preset minimum level. If the resolution of any video stream is not at the preset minimum level, the system sends a request to the server to switch the resolution of the corresponding video stream, and the video stream resolution in the request is the video resolution of the first preset level.

[0093] When adaptively adjusting the video resolution, the method also includes determining whether the resolution of each video stream displayed on the current playback interface is at the preset maximum level when the teacher or student is in the playback process of the merged video stream and the CPU utilization of the teacher or student is lower than the third preset value. If the resolution of any video stream is not at the preset maximum level, the teacher or student sends a request to the server to switch the resolution of the corresponding video stream, and the video stream resolution in the request is the video resolution of the second preset level.

[0094] This application also includes a content encoding adjustment device for AI online education, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement a content encoding adjustment method for AI online education.

[0095] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1.A method for adjusting content encoding of AI online education, characterized in that: The method is applied among a server, a teacher terminal and all student terminals, wherein: The teacher terminal and each student terminal encodes the captured video data and audio data to obtain original video streams of each terminal, and sends the original video streams to the server, and the server stores the related parameters of the original video streams in a mapping table; The server sends the received original video streams according to the number of original video streams requested by the teacher terminal or the student terminal, the teacher terminal and the student terminal play and display the received original video streams of each student terminal in a preset area in a page in the form of a preset window pane, and each window in the window pane plays and displays the original video stream of a student terminal; The teacher terminal or the student terminal monitors the frame rate of each original video stream in the current window pane played in real time, and when the frame rate is less than a first preset value, sends request information of a merged video stream to the server, and requests to merge the original video streams of multiple student terminals in the current window pane into one; The server calculates the ID number of the original video stream of the student terminal associated with the request information of the merged video stream according to the parameters of the request information, merges the original video streams of the student terminals with the associated ID number to obtain the corresponding merged video stream, and stores the related parameters of the merged video stream in the mapping table; The server sends a request response message of the merged video stream to the teacher terminal or the student terminal, and when the operation result of the request response message of the merged video stream is successful, sends the merged video stream with the corresponding parameters to the teacher terminal or the student terminal; The teacher terminal or the student terminal plays and displays the merged video stream in the current window pane after receiving the merged video stream; The teacher terminal or the student terminal also switches the layout type, switches the page code and switches the video stream resolution of the window pane playing and displaying the merged video stream, respectively. 2.The AI online education content encoding adjustment method of claim 1, wherein: The ID number sequence of all student end original video streams is ; For the original video stream of the student end playing the display in the pane of the teacher end or the student end, when there is no student speaking, the original video stream of the student end is sequentially and one-to-one displayed in each window according to the ID number sequence, and the sequence of displaying the original video stream of the student end in all windows is called a video stream sequence, and is represented as wherein represents the video stream corresponding to the th window in all windows of all panes, and ; When there is a student speaking, the original video stream position of the student terminal corresponding to the speaking student is interchanged with the first row and column window position in the window pane of the first page, and the server realizes the position interchanging of the two windows by regenerating the video stream of the window pane where the two interchanged window positions are located, and records the latest video stream sequence every time the position interchanging occurs. 3.The AI online education content encoding adjustment method of claim 2, wherein: When the teacher terminal or the student terminal switches the layout type, switches the page code or switches the video stream resolution of the window pane playing and displaying the merged video stream, respectively, the teacher terminal or the student terminal sends request information of window pane layout type switching, request information of window pane page code switching or request information of video stream resolution switching to the server, respectively; The server finds the corresponding merged video stream in the mapping table according to the window pane layout type, the window pane page code and the video stream resolution in the request information of window pane layout type switching, the request information of window pane page code switching or the request information of video stream resolution switching; When no corresponding merged video stream is found in the mapping table, the server calculates the ID number of the original video stream of the student terminal associated with the request information of the merged video stream according to the parameters of the request information of window pane layout type switching, the request information of window pane page code switching or the request information of video stream resolution switching, and the latest video stream sequence, and newly merges the original video streams of the student terminals with the associated ID number to obtain the corresponding merged video stream, and stores the related parameters of the merged video stream in the mapping table. When the corresponding merged video stream is found or after the latest merging, the server sends a request response message for switching the type of the window pane layout, a request response message for switching the page number of the window pane or a request response message for switching the resolution of the video stream to the teacher terminal or the student terminal, and when the operation result of the request response message for switching the type of the window pane layout, the request response message for switching the page number of the window pane or the request response message for switching the resolution of the video stream is successful, the server sends the corresponding parameter merged video stream to the teacher terminal or the student terminal. 4.The AI online education content encoding adjustment method of claim 3, characterized in that: The request information of the merged video stream, the request information for switching the type of the window pane layout, the request information for switching the page number of the window pane and the request information for switching the resolution of the video stream all include the parameters of the request information ID, the requester ID, the course instance ID, the operation type, the type of the window pane layout, the page number of the window pane and the resolution of the video stream. The request response message of the merged video stream, the request response message for switching the type of the window pane layout, the request response message for switching the page number of the window pane and the request response message for switching the resolution of the video stream all include the parameters of the response message ID, the requester ID, the course instance ID, the operation result, the merged video stream ID and the merged video address. 5.The AI online education content encoding adjustment method of claim 4, characterized in that: The server calculates the ID number of the original video stream of the student terminal associated with the request information according to the parameters in the request information of the merged video stream, the request information for switching the type of the window pane layout, the request information for switching the page number of the window pane or the request information for switching the resolution of the video stream, and merges the obtained original video stream of the student terminal associated with the ID number to obtain the corresponding merged video stream, and stores the related parameters of the merged video stream into a mapping table, including: The server determines the pane layout type based on the request information for merging video streams, switching pane layout types, switching pane page numbers, or switching video stream resolutions. and pane page numbers And based on the latest video stream order, calculate the original video stream ID number of the student terminal associated with the request information, and merge the original video streams of the student terminal with the obtained associated ID number to obtain a merged video; transcoding the resolution of the obtained merged video stream into the corresponding video resolution in the request information of the merged video stream, the request information for switching the type of the window pane layout, the request information for switching the page number of the window pane or the request information for switching the resolution of the video stream; generating the merged video stream ID and the merged video stream address for the transcoded merged video stream; establishing the mapping relationship between the merged video stream and the corresponding type of the window pane layout, the page number of the window pane, the resolution of the video stream, the merged video stream ID, the merged video stream address and the ID number of the original video stream of the student terminal associated with the merged video stream, and storing the mapping relationship into the mapping table, and for the original video stream, the mapping table stores the multiple gear video resolutions corresponding to each original video stream, multiple original video stream IDs and multiple original video stream addresses. 6.The AI online education content encoding adjustment method of claim 1, wherein: For the teacher terminal, the original video stream or the merged video stream of the student terminal received is displayed in the form of a window pane in the first area of the playing interface, and the original video stream of the teacher terminal received is displayed in the second area of the playing interface, and the playing interface is further provided with a third area for switching the type of the window pane layout of the first area, a fourth area for switching the page number of the window pane of the first area and a fifth area for switching the resolution of each video stream of the first area. For the student end, the received original video stream or the merged video stream of the student end is displayed in the form of a window pane in the sixth area of the playing interface, and the received original video stream of the teacher end is displayed in the seventh area of the playing interface. The playing interface further comprises an eighth area for switching the layout type of the window pane in the sixth area, a ninth area for switching the page number of the window pane in the sixth area, and a tenth area for switching the resolution of each video stream in the sixth area. 7.The AI online education content encoding adjustment method of claim 1, wherein: The AI online education content encoding adjustment method further comprises double-clicking the single-window video stream picture in the window pane displaying the merged video stream by the teacher end or the student end to realize zooming in or zooming out: When zooming in by double-clicking, first the teacher side or the student side computer counts the row and column numbers of the double-click event in the window pane, and the window pane layout type is , the top-left corner position coordinate of the window pane is , the bottom-right corner position coordinate of the window pane is , and the position coordinate of the double-click event is , then the row number of the double-click event in the window pane is , and the column number of the double-click event in the window pane is , wherein represents rounding down. When the calculation is completed, the teacher end or the student end sends a request information for zooming in the single-window video stream picture in the window pane to the server, and the request information for zooming in the single-window video stream picture in the window pane comprises a request information ID, a requester ID, a course instance ID, an operation type, a window pane layout type, a window pane page number, a zoomed-in video stream resolution, and a row and column number of the double-click event in the window pane; After the server receives the request information for zooming in the single-window video stream picture in the window pane, the server finds the original video stream ID number sequence of the merged video stream corresponding to the associated student end in the mapping table according to the window pane layout type, the window pane page number, and the zoomed-in video stream resolution in the request information, and obtains the ID number of the original video stream of the student end corresponding to the double-click event according to the row and column number of the double-click event in the window pane and the window pane layout type. Then, the server finds the original video stream ID and the original video stream address of the student end corresponding to the ID number and the original video stream resolution in the mapping table. The server sends a request response message for zooming in the single picture in the window pane to the teacher end or the student end, and the request response message comprises a response message ID, a requester ID, a course instance ID, an operation result, an original video stream ID of the zoomed-in single window, and an original video address. When the operation result in the response message is successful, the server sends the original video stream of the student end with the corresponding ID number and video stream resolution to the requester with the corresponding ID. After the requester with the corresponding ID receives the original video stream, the requester plays the original video stream in the form of covering the entire first area or the sixth area. 8.The AI online education content encoding adjustment method of claim 7, wherein: When the zoomed-in single-window video stream picture is double-clicked to zoom out, the student end or the teacher end sends a request information for double-clicking to zoom out the zoomed-in single-window video stream picture to the server, and the request information comprises a request information ID, a requester ID, a course instance ID, an operation type, a window pane layout type, a window pane page number, and a video stream resolution. After the server receives the request information for double-clicking to zoom out the zoomed-in single-window video stream picture, the server finds the corresponding merged video stream in the mapping table according to the window pane layout type, the window pane page number, and the video stream resolution in the request information. The server sends a request response message for double-clicking zooming out to the teacher end or the student end after zooming the single-window video stream picture, and the request response message comprises a response message ID, a requester ID, a course instance ID, an operation result, a merged video stream ID and a merged video address. When the operation result is successful, the server sends the merged video stream with corresponding parameters to the teacher end or the student end. 9.The AI online education content encoding adjustment method of claim 1, wherein: The content coding adjustment method of the AI online education further comprises adaptive adjustment of the video resolution. When the size of the playing interface of the teacher end or the student end is less than the second preset value, the teacher end or the student end judges whether the resolution of each video stream played and displayed by the current playing interface is the preset lowest resolution. When the resolution of a video stream is not in the preset lowest resolution, the teacher end or the student end sends a request information of video stream resolution switching to the server to request resolution adjustment of the corresponding video stream, and the video stream resolution in the request information is the preset lowest resolution. When the size of the playing interface of the teacher end or the student end is greater than or equal to the second preset value, the teacher end or the student end judges whether the resolution of each video stream played and displayed by the current playing interface is the preset highest resolution. When the resolution of a video stream is not in the preset highest resolution, the teacher end or the student end sends a request information of video stream resolution switching to the server to request resolution adjustment of the corresponding video stream, and the video stream resolution in the request information is the preset highest resolution. 10.The AI online education content encoding adjustment method of claim 9, wherein: In the adaptive adjustment of the video resolution, when the teacher end or the student end is in playing and buffering under playing of the merged video stream, the teacher end or the student end judges whether the resolution of each video stream played and displayed by the current playing interface is in the preset lowest resolution. When the resolution of a video stream is not in the preset lowest resolution, the teacher end or the student end sends a request information of video stream resolution switching to the server to request resolution adjustment of the corresponding video stream, and the video stream resolution in the request information is the first preset resolution. In the adaptive adjustment of the video resolution, when the teacher end or the student end is in playing and buffering under playing of the merged video stream, and the CPU usage rate of the teacher end or the student end is less than the third preset value, the teacher end or the student end judges whether the resolution of each video stream played and displayed by the current playing interface is in the preset highest resolution. When the resolution of a video stream is not in the preset highest resolution, the teacher end or the student end sends a request information of video stream resolution switching to the server to request resolution adjustment of the corresponding video stream, and the video stream resolution in the request information is the second preset resolution.

Images