Intelligent window adjustment method for AI online education

Abstract

The invention discloses an interaction method for AI online education, which is used for amplifying a video picture area related to a fixation point of a student. When the number of the student ends sending the request messages counted by the server is small, the whole request messages are directly calculated, and the corresponding server amplification video streams are generated and sent to the corresponding student ends, so that video picture areas related to fixation points of students on screens of the student ends are amplified, displayed and played, and the user experience is improved. The watching and learning effects of students are improved; when the number of student terminals sending the request message counted by the server is large, segmenting an original video picture of the server to form a small number of coding areas, generating a small number of coding video streams, extracting pixel data of the corresponding areas from the corresponding coding video streams to obtain corresponding server amplified video streams, and sending the amplified video streams to the server; a server amplification video stream does not need to be independently generated for each piece of request information, the server pressure is reduced, and the efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of online education technology, specifically relating to an AI-powered intelligent window adjustment method for online education. Background Technology

[0002] Currently, with the emergence of the mobile internet, the way knowledge is acquired has undergone fundamental changes. Teaching and learning can be conducted without the limitations of time, space, and location. Online education is becoming increasingly advantageous. Online education refers to a teaching model that uses the internet and other media. It breaks through the boundaries of time and space and is different from the traditional on-campus teaching model.

[0003] In online education scenarios, students attend classes via educational software on smart terminals. Initially, all students see the same area of ​​the video feed. However, each student focuses on different areas of the video feed at different times. The desired system is to automatically zoom in on the corresponding areas to improve the learning experience. Each student needs a stable, zoomed-in video stream related to their focus, and the system should automatically adjust the video feed area based on changes in focus. Current technologies typically require manual operation by students, such as clicking or swiping to select and zoom in on the desired area, which is cumbersome and negatively impacts the student's viewing experience. Summary of the Invention

[0004] The purpose of this invention is to address the problems mentioned in the background art by proposing a method for intelligent window adjustment in AI online education.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] This invention proposes an intelligent window adjustment method for AI-powered online education, applied to magnifying video frame areas related to students' gaze points, including:

[0007] All student terminals receive the remote teaching video stream sent by the server via multicast, display and play the video, and periodically obtain the coordinates of the student's gaze point with the upper left corner of the student's screen as the origin.

[0008] For each gaze point coordinate in each cycle, the corresponding student terminal determines whether the gaze point coordinate is within the video frame. If the gaze point coordinate is within the video frame, the gaze point coordinate is converted into a second coordinate point with the top left corner of the original video frame on the server as the origin.

[0009] Each student sends a request message to the server to zoom in on the video image containing the second coordinate point;

[0010] The server counts the number of student terminals sending request messages. When the number is less than the preset value, the server calculates the enlarged areas of the first video frames related to the second coordinate points in each request message in sequence, generates server-enlarged video streams corresponding to the enlarged areas of the first video frames, and sends them to the corresponding student terminals;

[0011] When the number is greater than or equal to the preset value, the server divides its original video frame into multiple divided areas;

[0012] The server determines the divided area to which each second coordinate point belongs according to the second coordinate points in the request message, and calculates the enlarged areas of the second video frames related to the second coordinate points in each video frame enlargement request message;

[0013] The server generates server-divided enlarged video streams corresponding to each divided area, which are called encoded video streams, and sends them to the corresponding student terminals. For each enlarged area of the second video frame, the student terminal intercepts a video stream of the corresponding area size from the corresponding encoded video stream to obtain the server-enlarged video stream corresponding to each enlarged area of the second video frame;

[0014] After each student terminal processes the server-enlarged video stream corresponding to the video frame enlarged area, it performs scaling display according to the size of its own video frame.

[0015] Preferably, the students in front of each student terminal watch the video frame by wearing eye movement tracking glasses, and each eye movement tracking glass periodically obtains the fixation point coordinates of the student with the upper left corner of the student terminal screen as the origin, and sends them to the corresponding student terminal.

[0016] Preferably, the student terminal determines whether the fixation point coordinates are within the video frame, including:

[0017] With the upper left corner of the student terminal screen as the origin, the fixation point coordinates of the student are expressed as (Std_Xt, Std_Yt);

[0018] With the upper left corner of the student terminal screen as the origin, the upper left corner coordinates of the video frame of the student terminal are obtained as (Std_X1, Std_Y1) and the lower right corner coordinates are (Std_X2, Std_Y2);

[0019] Determine whether the following conditions are met: Std_X1 < Std_Xt < Std_X2 and Std_Y1 < Std_Yt < Std_Y2. If they are met, the fixation point coordinates of the student are within the video frame of the student terminal, otherwise the fixation point coordinates of the student are not within the video frame of the student terminal.

[0020] Preferably, the conversion of the fixation point coordinates to the second coordinate points with the upper left corner of the original video frame of the server as the origin includes:

[0021] First, convert the fixation point coordinates to the first coordinate point (Std_Pic_Xt, Std_Pic_Yt) with the upper left corner of the video screen on the student side as the origin, where (Std_Pic_Xt, Std_Pic_Yt) = (Std_Xt - Std_X1, Std_Yt - Std_Y1);

[0022] Then, convert the first coordinate point to the second coordinate point (Srv_Xt, Srv_Yt) with the upper left corner of the original video screen on the server as the origin, and Srv_Xt = floor(Std_Pic_Xt / Std_ScaleX), Srv_Yt = floor(Std_Pic_Yt / Std_ScaleY), where floor() represents rounding down, Std_ScaleX represents the scaling ratio of the width of the video screen on the student side to the original video screen on the server, and Std_ScaleY represents the scaling ratio of the height of the video screen on the student side to the original video screen on the server;

[0023] Among them, when taking the upper left corner of the video screen on the student side as the origin, the corresponding coordinate system is: starting from the origin, the positive direction of the x-axis is along the width direction of the video screen on the student side, and the positive direction of the y-axis is along the height direction of the video screen on the student side;

[0024] When taking the upper left corner of the original video screen on the server as the origin, the corresponding coordinate system is: starting from the origin, the positive direction of the x-axis is along the width direction of the original video screen on the server, and the positive direction of the y-axis is along the height direction of the original video screen on the server.

[0025] Preferably, each request message sent by the student side includes a message ID, the ID of the student side, the IP address of the student side, the second coordinate point of the student side, and a request timestamp.

[0026] Preferably, when the quantity is less than the preset value, the server calculates the first video screen magnification area related to the second coordinate point in each request message in sequence, including:

[0027] The server sets the width of the first video screen magnification area and the second video screen magnification area related to each second coordinate point to be uniformly Area_W, and the height to be uniformly Area_H, and Area_W = a * Srv_W, Area_H = a * Srv_H, where 0 < a < 1, Srv_W is the width of the original video screen on the server, and Srv_H is the height of the original video screen on the server;

[0028] Taking the upper left corner of the original video screen on the server as the origin, calculate the center point coordinates of the first video screen magnification area corresponding to each student side according to the second coordinate point, the width and height of the first video screen magnification area, and the width and height of the original video screen on the server:

[0029] When the first condition Area_W / 2 <= Srv_Xt <= Srv_W - Area_W / 2 and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2 is satisfied, the enlarged area of the first video frame is completely within the original server frame. At this time, the center point coordinates (Srv_Xo, Srv_Yo) of the enlarged area of the first video frame are the second coordinate point (Srv_Xt, Srv_Yt).

[0030] When the first condition is not satisfied, the enlarged area of the first video frame is not completely within the original server frame. At this time, it is necessary to adjust the enlarged area of the first video frame so that the area is completely within the original server frame, and the calculation of the center point coordinates of the adjusted enlarged area of the first video frame is as follows:

[0031] When 0 < Srv_Xt < Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Area_H / 2).

[0032] When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_Xt, Area_H / 2).

[0033] When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Area_H / 2).

[0034] When Srv_W - Area_W / 2 < Srv_Xt < Srv_W and Area_H / 2 <= Srv_Yt < Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_H - Area_H / 2, Srv_Yt).

[0035] When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W, and Srv_H - Area_H / 2 <= Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Srv_H - Area_H / 2);

[0036] When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2, and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_Xt, Srv_H - Area_H / 2);

[0037] When 0 < Srv_Xt < Area_W / 2, and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_H - Area_H / 2);

[0038] When 0 < Srv_Xt < Area_W / 2, and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_Yt);

[0039] According to the center point coordinates of the enlarged area of the first video frame and the width and height of the enlarged area of the first video frame, the four - corner coordinates of the enlarged area of the first video frame with the upper - left corner of the original video frame of the server as the origin are calculated.

[0040] Preferably, generating a server - enlarged video stream corresponding to the enlarged area of the first video frame includes:

[0041] According to the four - corner coordinates of the enlarged area of the first video frame with the upper - left corner of the original video frame of the server as the origin, extracting the pixel data of the corresponding area from the server original video stream, and using an image enlargement algorithm to enlarge the pixel data of the extracted area by 1 / a times proportionally to obtain the server - enlarged video stream corresponding to the enlarged area of the first video frame.

[0042] Preferably, when the quantity is greater than or equal to a preset value, when the server performs regional segmentation on its original video picture, first the server divides its original video picture, divides the original video picture into m equal parts along both the width and the height, then divides the original video picture into m*m squares, and obtains (m + 1)*(m + 1) coordinate points on the original video picture. Among the (m + 1)*(m + 1) coordinate points, the number of coordinate points located on the boundary of the original video picture is 4m, the number of coordinate points located inside the original video picture is (m - 1)*(m - 1), and the coordinate points located inside the original video picture are called the third coordinate points;

[0043] Set the width of each segmentation area to be uniformly Srv_W’, and the height to be uniformly Srv_H’, and Srv_W’ = b*Srv_W, Srv_H’ = b*Srv_H, where 0 < b < 1, and b > a, b = (m - n) / m, n < m, and n is a positive integer, and there is an overlapping part between each segmentation area, and then obtain multiple segmentation areas from the server's original video picture;

[0044] The server calculates the distance between the point where each second coordinate point is located and the third coordinate point respectively, selects the nearest third coordinate point for the point where each second coordinate point is located, and determines the segmentation area to which each selected third coordinate point belongs, and regards the boundary where the third coordinate point is located in the segmentation area as not belonging to the segmentation area, and takes the segmentation area to which it belongs as the segmentation area to which the second coordinate point belongs, and the segmentation area to which it belongs is called the coding area. When there are multiple segmentation areas where the selected third coordinate point is located, randomly select one segmentation area as the coding area, and establish a mapping table between all second coordinate points and the belonging coding area;

[0045] Taking the upper left corner of the server's original video picture as the origin, obtain the upper left corner coordinates of each coding area as (Srv_Sca_Xt, Srv_Sca_Yt), and convert the second coordinate point into a fourth coordinate point (Srv_Xt’, Srv_Yt’) = (Srv_Xt - Srv_Sca_Xt, Srv_Yt - Srv_Sca_Yt) with the upper left corner coordinates of the corresponding coding area as the origin;

[0046] Taking the upper left corner coordinates of the coding area as the origin, calculate the center point coordinates of the second video picture magnification area corresponding to each student terminal according to the fourth coordinate point, the width and height of the second video picture magnification area, and the width and height of the coding area, and the width and height of the coding area are Srv_W’ and Srv_H’ respectively.

[0047] Preferably, the calculating the center point coordinates of the second video picture magnification area corresponding to each student terminal includes:

[0048] When the second condition Area_W / 2 <= Srv_Xt’ <= Srv_W’ - Area_W / 2 and Area_H / 2 <= Srv_Yt’ <= Srv_H’ - Area_H / 2 is satisfied, the enlarged area of the second video frame is completely within the corresponding encoded area. At this time, the center point coordinates (Srv_Xo’, Srv_Yo’) of the enlarged area of the second video frame are the fourth coordinate point (Srv_Xt’, Srv_Yt’).

[0049] When the second condition is not satisfied, the enlarged area of the second video frame is not completely within the corresponding encoded area. At this time, it is necessary to adjust the enlarged area of the second video frame so that the area is completely within the corresponding encoded area, and the calculation of the center point coordinates of the adjusted enlarged area of the second video frame is as follows:

[0050] When 0 < Srv_Xt’ < Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Area_H / 2);

[0051] When Area_W / 2 <= Srv_Xt’ < Srv_W’ - Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Area_H / 2);

[0052] When Srv_W’ - Area_W / 2 <= Srv_Xt’ < Srv_W’ and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_W’ - Area_W / 2, Area_H / 2);

[0053] When Srv_W’ - Area_W / 2 < Srv_Xt’ < Srv_W’ and Area_H / 2 <= Srv_Yt’ < Srv_H’ - Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_H’ - Area_H / 2, Srv_Yt’);

[0054] When Srv_W’ - Area_W / 2 <= Srv_Xt’ < Srv_W’, and Srv_H’ - Area_H / 2 <= Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged Area of the second video Image are (Srv_Xo’, Srv_Yo’) = (Srv_W’ - Area_W / 2, Srv_H’ - Area_H / 2);

[0055] When Area_W / 2 <= Srv_Xt’ < Srv_W’ - Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged Area of the second video Image are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Srv_H’ - Area_H / 2);

[0056] When 0 < Srv_Xt’ < Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged Area of the second video Image are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_H’ - Area_H / 2);

[0057] When 0 < Srv_Xt’ < Area_W / 2, and Area_H / 2 <= Srv_Yt’ <= Srv_H’ - Area_H / 2, the center point coordinates of the enlarged Area of the second video Image are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_Yt’);

[0058] Based on the center point coordinates of the enlarged Area of the second video Image and the width and height of the enlarged Area of the second video Image, calculate the four corner coordinates of the enlarged Area of the second video Image with the upper left corner of the encoding Area as the origin, and convert these four corner coordinates to the four corner coordinates with the upper left corner of the server's original video Image as the origin.

[0059] Preferably, the process of generating the server - enlarged video stream corresponding to the enlarged Area of the second video Image includes:

[0060] The server extracts the pixel data of the corresponding areas of each encoding Area from the server's original video stream according to the positions and widths and heights of each encoding Area, and uses an image enlargement algorithm to enlarge the pixel data of the extracted areas by 1 / b times in proportion to obtain the encoding video streams corresponding to each encoding Area;

[0061] According to the mapping table, find the encoded video stream corresponding to the second video stream magnification area related to each second coordinate point, and send the response information containing the corresponding encoded video stream and the four corner coordinates of the second video magnification area with the top left corner of the server's original video screen as the origin to the corresponding student terminal.

[0062] After receiving the response information, the student terminal extracts the pixel data of the corresponding area from the corresponding encoded video stream based on the four corner coordinates of the second video stream magnification area with the top left corner of the server's original video screen as the origin, and obtains the server magnified video stream corresponding to the magnified area of ​​the second video screen.

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] When the number of student terminals sending request messages counted by the server is small, the intelligent window adjustment method of this AI online education directly calculates and generates a corresponding enlarged video stream to the server and sends it to the corresponding student terminal. This enables the enlarged display and playback of the video screen area related to the student's gaze point on the student's terminal screen, thereby improving the student's viewing and learning effect.

[0065] When the number of student clients sending request messages counted by the server is large, the original video frame of the server is segmented into a small number of encoded regions, and a small number of encoded video streams are generated. Then, according to the mapping relationship between the second coordinate point and its corresponding encoded region in the mapping table, the encoded video stream corresponding to each second coordinate point is obtained. Finally, the pixel data of the corresponding region is extracted from the corresponding encoded video stream directly according to the four corner coordinates of the magnified area of ​​the second video stream to obtain the corresponding server magnified video stream. There is no need to generate a server magnified video stream separately for each request, which reduces server load and improves efficiency. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating the intelligent window adjustment method for AI online education according to the present invention.

[0067] Figure 2 This is a schematic diagram of the original video frame after being divided by the server of this invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] In one embodiment, such as Figures 1-2As shown, an AI-powered online education window intelligent adjustment method is provided, applied to magnify the video frame area related to the student's gaze point (this application improves the student's learning effect by magnifying the video frame area related to the student's gaze point, allowing the student to view the video stream corresponding to the magnified area), including:

[0070] Step 1: All student terminals receive the remote teaching video stream sent by the server via multicast (it should be noted that the teacher sends the teaching video stream via the server in a multicast manner, and multiple students join the multicast group through the educational software client on their smart terminals (referred to as student terminals) to watch the teaching video stream), display and play the video screen, and periodically obtain the coordinates of the student's gaze point with the upper left corner of the student's screen as the origin.

[0071] Students at each student terminal watch the video by wearing eye-tracking glasses. Each eye-tracking glasses periodically acquires the student's gaze coordinates with the upper left corner of the student's screen as the origin. (The eye-tracking glasses periodically acquire gaze coordinates to follow changes in the student's eyeballs and acquire different gaze coordinates. The eye-tracking glasses capture the light reflected from the eyeballs through hardware devices (such as infrared light sources and cameras), combine pupil-corneal reflection methods or radio frequency signal technology, use algorithms to analyze the direction of the gaze, and map the eyeball movement data to the screen coordinate system to calculate the user's gaze coordinates on the screen in real time. This process is existing technology.) and sends the data to the corresponding student terminal.

[0072] When the top left corner of the student's screen is taken as the origin, the corresponding coordinate system is as follows: the positive x-axis starts from the origin along the width of the student's screen, and the positive y-axis starts from the origin along the height of the student's screen.

[0073] Step 2: For each gaze point coordinate in each cycle, the corresponding student terminal determines whether the gaze point coordinate is within the video frame. If the gaze point coordinate is within the video frame, the gaze point coordinate is converted into a second coordinate point with the top left corner of the original video frame on the server as the origin.

[0074] The student-side determination of whether the gaze point coordinates are within the video frame includes:

[0075] With the top left corner of the student's screen as the origin, the coordinates of the student's gaze point are represented as (Std_Xt, Std_Yt).

[0076] Taking the upper left corner of the student - side screen as the origin, (for the student side) obtain the upper left corner coordinates (Std_X1, Std_Y1) and the lower right corner coordinates (Std_X2, Std_Y2) of the video image of the student side (that is, the video image showing the played video). Specifically, how to obtain the upper left corner coordinates and the lower right corner coordinates of the video image of the student side is the docking between the student side and its operating system, which belongs to the prior art and does not fall within the protection scope of this application;

[0077] Determine whether the following conditions are met: Std_X1 < Std_Xt < Std_X2 and Std_Y1 < Std_Yt < Std_Y2. If so, the gaze point coordinates of the student are within the video image of the student side; otherwise, the gaze point coordinates of the student are not within the video image of the student side (when the gaze point coordinates of the student are not within the video image of the student side, no processing is required, and continue to judge the gaze point coordinates obtained in the next cycle).

[0078] Among them, converting the gaze point coordinates to the second coordinate point with the upper left corner of the original video image of the server as the origin includes:

[0079] First, convert the gaze point coordinates to the first coordinate point (Std_Pic_Xt, Std_Pic_Yt) = (Std_Xt - Std_X1, Std_Yt - Std_Y1) with the upper left corner of the video image of the student side as the origin; when taking the upper left corner of the video image of the student side as the origin, the corresponding coordinate system is: starting from the origin, the positive direction of the x - axis is along the width direction of the video image of the student side, and the positive direction of the y - axis is along the height direction of the video image of the student side;

[0080] The first coordinate point is then converted to a second coordinate point (Srv_Xt, Srv_Yt) with the top-left corner of the server's original video frame as the origin. Srv_Xt = floor(Std_Pic_Xt / Std_ScaleX), Srv_Yt = floor(Std_Pic_Yt / Std_ScaleY), where floor() represents rounding down, Std_ScaleX represents the scaling ratio in width between the student's video frame and the server's original video frame, and Std_ScaleY represents the scaling ratio in height between the student's video frame and the server's original video frame. Furthermore, Std_ScaleX = Std_W / S rv_W, Std_ScaleY=Std_H / Srv_H, where Srv_W*Srv_H represents the resolution of the server's original video frame (i.e., the width multiplied by the height of the server's original video frame, such as 1920*1080, in pixels), and Std_W*Std_H represents the resolution of the student's video frame (i.e., the width multiplied by the height of the student's video frame, such as 1280*720, in pixels). When the top-left corner of the server's original video frame is taken as the origin, the corresponding coordinate system is: the positive x-axis is along the width direction of the server's original video frame, and the positive y-axis is along the height direction of the server's original video frame, starting from the origin.

[0081] Step 3: Each student terminal sends a request message to the server to zoom in on the video image containing the second coordinate point. The server counts the number of student terminals that send request messages. When the number is less than a preset value (i.e., when the number of student terminals that send request messages is small), then step 4 is executed. When the number is greater than or equal to the preset value (i.e., when the number of student terminals that send request messages is large), then step 5 is executed.

[0082] The request message sent by each student terminal includes a message ID, the student terminal ID, the student terminal IP address, the student terminal's second coordinate point, and the request timestamp.

[0083] Step 4: The server sequentially calculates the magnified area of ​​the first video frame related to the second coordinate point in each request message, generates a server magnified video stream corresponding to each magnified area of ​​the first video frame, and sends it to the corresponding student terminal.

[0084] The server sequentially calculates the magnified area of ​​the first video frame related to the second coordinate point in each request message, including:

[0085] The server sets the width of the first video frame magnification area and the second video frame magnification area related to each second coordinate point to be uniformly Area_W, and the height to be uniformly Area_H (this height and width are the height and width of a single frame in the video frame), and Area_W = a * Srv_W, Area_H = a * Srv_H, where 0 < a < 1 (such as a = 0.5), Srv_W is the width of the original video frame of the server, and Srv_H is the height of the original video frame of the server;

[0086] Taking the upper left corner of the original video frame of the server as the origin, (the server) calculates the center point coordinates of the first video frame magnification area corresponding to each student terminal according to the second coordinate point, the width and height of the first video frame magnification area, and the width and height of the original video frame of the server:

[0087] When the first condition Area_W / 2 <= Srv_Xt <= Srv_W - Area_W / 2 and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2 is satisfied, the first video frame magnification area is completely within the original server screen (where when a side of the first video frame magnification area exactly coincides with a side of the original server screen, this situation is regarded as the side of the first video frame magnification area being within the original server screen). At this time, the center point coordinates (Srv_Xo, Srv_Yo) of the first video frame magnification area are the second coordinate point (Srv_Xt, Srv_Yt);

[0088] When the first condition is not satisfied, the first video frame magnification area is not completely within the original server screen. At this time, it is necessary to adjust the first video frame magnification area so that the area is completely within the original server screen, and the calculation of the center point coordinates of the adjusted first video frame magnification area is as follows (obtain the adjusted first video frame magnification area by calculating this center point coordinate):

[0089] When 0 < Srv_Xt < Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the first video frame magnification area are (Srv_Xo, Srv_Yo) = (Area_W / 2, Area_H / 2);

[0090] When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the first video frame magnification area are (Srv_Xo, Srv_Yo) = (Srv_Xt, Area_H / 2);

[0091] When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Area_H / 2);

[0092] When Srv_W - Area_W / 2 < Srv_Xt < Srv_W and Area_H / 2 <= Srv_Yt < Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_H - Area_H / 2, Srv_Yt);

[0093] When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W and Srv_H - Area_H / 2 <= Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Srv_H - Area_H / 2);

[0094] When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2 and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_Xt, Srv_H - Area_H / 2);

[0095] When 0 < Srv_Xt < Area_W / 2 and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_H - Area_H / 2);

[0096] When 0 < Srv_Xt < Area_W / 2 and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_Yt);

[0097] Based on the center coordinates of the magnified area of ​​the first video frame and its width and height, the coordinates of the four corners of the magnified area of ​​the first video frame, with the top left corner of the original video frame on the server as the origin, are calculated. The top left corner coordinates are (Srv_Xt1, Srv_Yt1) = (Srv_Xo - Aera_W / 2, Srv_Yo - Aera_H / 2), and the top right corner coordinates are (Srv_Xt2, Srv_Yt2) = (Srv_Xt1 - Aera_W / 2, Srv_Yo - Aera_H / 2). Given v_Xo+Aera_W / 2, Srv_Yo-Aera_H / 2, the lower left corner coordinates are (Srv_Xt3, Srv_Yt3) = (Srv_Xo-Aera_W / 2, Srv_Yo+Aera_H / 2), and the lower right corner coordinates are (Srv_Xt4, Srv_Yt4) = (Srv_Xo+Aera_W / 2, Srv_Yo+Aera_H / 2), the magnified area of ​​the first video image related to the second coordinate point can be obtained.

[0098] The process of generating a server-enlarged video stream corresponding to the enlarged area of ​​the first video image includes:

[0099] Based on the calculated coordinates of the four corners of the first video frame magnification area with the top left corner of the server's original video frame as the origin, the pixel data of the corresponding area is extracted from the server's original video stream (i.e., the pixel data of the corresponding area of ​​each frame in the server's original video stream is extracted). The extracted pixel data is magnified by 1 / a times using an image magnification algorithm (such as bilinear interpolation) to obtain the server magnified video stream corresponding to the first video frame magnification area (and then sent to the corresponding student terminal). Then, step 6 is executed.

[0100] Step 5: Step 5.1, the server performs region segmentation on its original video frame, forming multiple segmented regions.

[0101] When the server performs region segmentation on its original video frame, it first divides the original video frame into m equal parts along both its width and height (e.g., four equal parts, e.g., ...). Figure 2As shown, taking m = 4 as an example, with the upper left corner of the original video frame of the server as the origin, the horizontal right direction as the positive direction of the X-axis, and the vertical downward direction as the positive direction of the Y-axis), the original video frame is divided into m * m squares, and (m + 1) * (m + 1) coordinate points are obtained on the original video frame. Among the (m + 1) * (m + 1) coordinate points, the number of coordinate points located on the boundary of the original video frame is 4m, and the number of coordinate points located inside the original video frame is (m - 1) * (m - 1). And the coordinate points located inside the original video frame are called the third coordinate points (where all the third coordinate points are sorted in sequence as I1, I2,..., Ik,..., I(m - 1) * (m - 1) in the order of first from left to right and then from top to bottom), and the coordinates of Ik are (((k - 1) mod (m - 1)+1) * Srv_W / m, (floor((k - 1) / (m - 1))+1) * Srv_H / m), mod() represents taking the remainder;

[0102] Set the width of each divided area to be uniformly Srv_W’, and the height to be uniformly Srv_H’, and Srv_W’ = b * Srv_W, Srv_H’ = b * Srv_H, where 0 < b < 1, and b > a, b = (m - n) / m, n < m, and n is a positive integer. In this embodiment, b = (m - 1) / m, and there are overlapping parts between the divided areas, and then multiple divided areas are obtained from the original video frame of the server; where as Figure 2 shown, when m = 4 and n = 1, the (m + 1) * (m + 1) coordinate points on the original video frame are sorted in sequence as P0 (which is the origin), T1, T2, T3, T4, L1, I1, I2, I3, R1, L2, I4, I5, I6, R2, L3, I7, I8, I9, R3, L4, B1, B2, B3, B4 in the order of first from left to right and then from top to bottom. Then there are four divided areas in the original video frame of the server, which are the area formed by points P0, T3, I9, and L3 (which can be simplified as P0T3I9L3), the area formed by points T1, T4, R3, and I7 (which can be simplified as T1T4R3I7), the area formed by points L1, I3, B3, and L4 (which can be simplified as L1I3B3L4), and the area formed by points I1, R1, B4, and B1 (which can be simplified as I1R1B4B1).

[0103] Step 5.2: The server determines the divided area to which each second coordinate point belongs according to the second coordinate points in the request message, and calculates the second video frame magnification area related to the second coordinate points in each video frame magnification request message.

[0104] The server calculates the distance between each second coordinate point and the third coordinate point, selects the nearest third coordinate point for each second coordinate point, and determines the segmentation region to which each selected third coordinate point belongs. A third coordinate point located on the boundary of a segmentation region is considered not to belong to that region (i.e., if a third coordinate point falls on the boundary of the current segmentation region, it does not belong to the current segmentation region). This segmentation region is then designated as the segmentation region to which the second coordinate point belongs, and this segmentation region is called the encoding region. When multiple segmentation regions exist for a selected third coordinate point, one is randomly selected as the encoding region. A mapping table is established between all second coordinate points and their respective encoding regions (the mapping table contains the second coordinate point and its encoding region, where the encoding region is the coordinates of its four corners). Figure 2 For example, in the original video frame of the server, the second coordinate points are A1, A2, A3, A4, and A5. Points A1 and A2 are closest to points I1 and I2, respectively, and points I1 and I2 are both located within the segmentation region P0T3I9L3. Therefore, the encoding region for points A1 and A2 is P0T3I9L3. Points A3 and A4 are closest to points I6 and I9, respectively, and points I6 and I9 are both located within the segmentation region I1R1B4B1 (Note: Point I6 can also be considered to be located within the segmentation region T1T4R3I7; either one can be selected). Therefore, the encoding region for points A3 and A4 is I1R1B4B1. Point A5 is closest to point I7, and point I7 is located within the segmentation region L1I3B3L4. Therefore, the encoding region for point A5 is L1I3B3L4 (the four corner coordinates of each encoding region can be directly derived and are not shown in this application).

[0105] Using the top-left corner of the original video frame on the server as the origin, obtain the top-left corner coordinates of each encoded region as (Srv_Sca_Xt, Srv_Sca_Yt). Convert the second coordinate point into a fourth coordinate point (Srv_Xt', Srv_Yt') = (Srv_Xt-Srv_Sca_Xt, Srv_Yt-Srv_Sca_Yt) with the top-left corner coordinates of the corresponding encoded region as the origin.

[0106] Taking the top-left corner of the encoding area as the origin, and based on the fourth coordinate point, the width and height of the second video zoom-in area, and the width and height of the encoding area, with the width and height of the encoding area being Srv_W' and Srv_H' respectively, calculate the center point coordinates of the second video zoom-in area corresponding to each student's terminal, including:

[0107] When the second condition Area_W / 2 <= Srv_Xt’ <= Srv_W’-Area_W / 2 and Area_H / 2 <= Srv_Yt’ <= Srv_H’-Area_H / 2 is satisfied, the enlarged area of the second video frame is completely within the corresponding coding area (where if a side of the enlarged area of the second video frame exactly coincides with a side of the coding area, this situation is considered that the side of the enlarged area of the second video frame is within the coding area). At this time, the center point coordinates (Srv_Xo’, Srv_Yo’) of the enlarged area of the second video frame are the fourth coordinate point (Srv_Xt’, Srv_Yt’).

[0108] When the second condition is not satisfied, the enlarged area of the second video frame is not completely within the corresponding coding area. At this time, it is necessary to adjust the enlarged area of the second video frame so that the area is completely within the corresponding coding area, and the calculation of the center point coordinates of the adjusted enlarged area of the second video frame is as follows (obtain the adjusted enlarged area of the second video frame by calculating the center point coordinates):

[0109] When 0 < Srv_Xt’ < Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Area_H / 2);

[0110] When Area_W / 2 <= Srv_Xt’ < Srv_W’- Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Area_H / 2);

[0111] When Srv_W’- Area_W / 2 <= Srv_Xt’ < Srv_W’ and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_W’- Area_W / 2, Area_H / 2);

[0112] When Srv_W’- Area_W / 2 < Srv_Xt’ < Srv_W’ and Area_H / 2 <= Srv_Yt’ < Srv_H’-Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_H’-Area_H / 2, Srv_Yt’).

[0113] When Srv_W’ - Area_W / 2 <= Srv_Xt’ < Srv_W’, and Srv_H’ - Area_H / 2 <= Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_W’ - Area_W / 2, Srv_H’ - Area_H / 2).

[0114] When Area_W / 2 <= Srv_Xt’ < Srv_W’ - Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Srv_H’ - Area_H / 2).

[0115] When 0 < Srv_Xt’ < Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_H’ - Area_H / 2).

[0116] When 0 < Srv_Xt’ < Area_W / 2, and Area_H / 2 <= Srv_Yt’ <= Srv_H’ - Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_Yt’);

[0117] The center point coordinates of the enlarged area of the second video frame above are all based on the upper left corner of the encoded area as the origin;

[0118] The server calculates the coordinates of the four corners of the enlarged area of ​​the second video frame, with the top-left corner of the encoded area as the origin, based on the center point coordinates and the width and height of the enlarged area. Specifically, with the top-left corner of the encoded area as the origin, the coordinates of the top-left corner of the enlarged area are (Srv_Xlt', Srv_Ylt') = (Srv_Xo' - Aera_W / 2, Srv_Yo' - Aera_H / 2), and the coordinates of the top-right corner are (Srv_Xt', Srv_Ylt') = (Srv_Xo' - Aera_W / 2, Srv_Yo' - Aera_H / 2). 2, Srv_Yt2) = (Srv_Xo' + Aera_W / 2, Srv_Yo' - Aera_H / 2), the coordinates of the lower left corner are (Srv_Xt3, Srv_Yt3) = (Srv_Xo' - Aera_W / 2, Srv_Yo' + Aera_H / 2), and the coordinates of the lower right corner are (Srv_Xrb', Srv_Yrb') = (Srv_Xo' + Aera_W / 2, Srv_Yo' + Aera_H / 2);

[0119] The server converts the four-corner coordinates of the second video image magnification area, with the top left corner of the encoded area as the origin, to the four-corner coordinates of the original video image on the server, which is done by multiplying each of the four-corner coordinates of the second video image magnification area with the top left corner of the encoded area as the origin by 1 / b, thereby obtaining the second video image magnification area related to the second coordinate point.

[0120] Step 5.3: The server generates server-segmented magnified video streams corresponding to each segmented region, referred to as encoded video streams, and sends them to the corresponding student terminals. For each magnified region of the second video frame, the student terminal extracts a video stream of the corresponding region size from the corresponding encoded video stream to obtain the server-magnified video stream corresponding to each magnified region of the second video frame, including:

[0121] Based on the location and dimensions of each encoding region, the server extracts the pixel data corresponding to each encoding region (for each frame) from the original video stream. Then, it uses an image magnification algorithm (such as bilinear interpolation) to magnify the extracted pixel data by a factor of 1 / b, thus obtaining the encoded video stream corresponding to each encoding region (for repeated encoding regions, it only needs to be generated once).

[0122] According to the mapping table, the server finds the encoded video stream corresponding to the second video stream magnification area related to each second coordinate point (that is, according to the mapping relationship between the second coordinate point and its corresponding encoding area in the mapping table, the server first finds the encoding area corresponding to the second coordinate point, and then obtains the corresponding encoded video stream), and sends response information containing the corresponding encoded video stream and the four corner coordinates of the second video image magnification area with the top left corner of the server's original video image as the origin to the corresponding student terminal.

[0123] After receiving the response information, the student terminal extracts the pixel data of the corresponding area (i.e., the four-corner coordinates of the second video stream magnification area with the top left corner of the server's original video frame as the origin, for each frame) from the corresponding encoded video stream based on the four-corner coordinates of the second video stream magnification area with the top left corner of the server's original video frame as the origin, and obtains the server magnified video stream corresponding to the second video stream magnification area, and then executes step 6.

[0124] Step 6: After each student terminal zooms in on the video stream corresponding to the zoomed-in area of ​​the video screen (either the first or second zoomed-in area), it scales the display according to its own video screen size (i.e., scales it to the resolution Std_W*Std_H). The student terminal then zooms in on the video screen area related to the student's gaze point on the student's screen.

[0125] In this application, * indicates multiplication.

[0126] In another embodiment, this application also includes a window intelligent adjustment device for AI online education, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the method in steps 1-6. Specific limitations regarding the window intelligent adjustment device for AI online education can be found in the limitations of the window intelligent adjustment method for AI online education described above, and will not be repeated here.

[0127] When the number of student terminals sending request messages counted by the server is small, the intelligent window adjustment method of this AI online education directly calculates and generates a corresponding enlarged video stream to the server and sends it to the corresponding student terminal. This enables the enlarged display and playback of the video screen area related to the student's gaze point on the student's terminal screen, thereby improving the student's viewing and learning effect.

[0128] When the number of student clients sending request messages counted by the server is large, the original video frame of the server is segmented into a small number of encoded regions, and a small number of encoded video streams are generated. Then, according to the mapping relationship between the second coordinate point and its corresponding encoded region in the mapping table, the encoded video stream corresponding to each second coordinate point is obtained. Finally, the pixel data of the corresponding region is extracted from the corresponding encoded video stream directly according to the four corner coordinates of the magnified area of ​​the second video stream to obtain the corresponding server magnified video stream. There is no need to generate a server magnified video stream separately for each request, which reduces server load and improves efficiency.

[0129] It should be understood that, although Figure 1The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for intelligent window adjustment in AI-powered online education, characterized in that: Applied to magnify the video picture area related to the student's fixation point, including: All student terminals receive the remote teaching video stream multicast by the server, display and play the video picture, and periodically obtain the fixation point coordinates of the student with the upper left corner of the student terminal screen as the origin; For each fixation point coordinate in each cycle, the corresponding student terminal determines whether the fixation point coordinate is within the video picture. When the fixation point coordinate is within the video picture, the fixation point coordinate is converted into a second coordinate point with the upper left corner of the server's original video picture as the origin; Each student terminal sends a request message for magnifying the video picture containing the second coordinate point to the server; The server counts the number of student terminals sending the request message. When the number is less than the preset value, the server calculates the first video picture magnification area related to the second coordinate point in each request message in turn, generates a server magnified video stream corresponding to each first video picture magnification area, and sends it to the corresponding student terminal; When the number is greater than or equal to the preset value, the server divides its original video picture into multiple divided areas; The server determines the divided area to which each second coordinate point belongs according to the second coordinate point in the request message, and calculates the second video picture magnification area related to the second coordinate point in each video picture magnification request message; The server generates a server divided and magnified video stream corresponding to each divided area, which is called an encoded video stream, and sends it to the corresponding student terminal. For each second video picture magnification area, the student terminal intercepts a video stream with the corresponding area size from the corresponding encoded video stream to obtain a server magnified video stream corresponding to each second video picture magnification area; After each student terminal magnifies the server magnified video stream corresponding to the video picture magnification area, it performs scaling display according to the size of its own video picture.

2. The AI ​​online education window intelligent adjustment method as described in claim 1, characterized in that: The students in front of each student terminal watch the video picture by wearing eye tracking glasses, and each eye tracking glasses periodically obtains the fixation point coordinates of the student with the upper left corner of the student terminal screen as the origin and sends them to the corresponding student terminal.

3. The AI ​​online education window intelligent adjustment method as described in claim 1, characterized in that: The student terminal determines whether the fixation point coordinate is within the video picture, including: With the upper left corner of the student terminal screen as the origin, the fixation point coordinates of the student terminal are expressed as (Std_Xt, Std_Yt); With the upper left corner of the student terminal screen as the origin, the upper left corner coordinates of the video picture of the student terminal are obtained as (Std_X1, Std_Y1) and the lower right corner coordinates are obtained as (Std_X2, Std_Y2); Determine whether it satisfies: Std_X1 < Std_Xt < Std_X2 and Std_Y1 < Std_Yt < Std_Y2. If it satisfies, the fixation point coordinates of the student are within the video picture of the student terminal, otherwise the fixation point coordinates of the student are not within the video picture of the student terminal.

4. The AI ​​online education window intelligent adjustment method as described in claim 3, characterized in that: The conversion of the fixation point coordinate into a second coordinate point with the upper left corner of the server's original video picture as the origin includes: First, the fixation point coordinate is converted into a first coordinate point (Std_Pic_Xt, Std_Pic_Yt) = (Std_Xt - Std_X1, Std_Yt - Std_Y1) with the upper left corner of the student terminal's video picture as the origin; The first coordinate point is converted into a second coordinate point (Srv_Xt, Srv_Yt) with the upper left corner of the original video frame of the server as the origin, and Srv_Xt = floor(Std_Pic_Xt / Std_ScaleX), Srv_Yt = floor(Std_Pic_Yt / Std_ScaleY), where floor() represents rounding down, Std_ScaleX represents the scaling ratio of the video frame of the student side to the original video frame of the server in width, and Std_ScaleY represents the scaling ratio of the video frame of the student side to the original video frame of the server in height; Among them, when the upper left corner of the video frame of the student side is used as the origin, the corresponding coordinate system is: starting from the origin, the positive direction of the x-axis is along the width direction of the video frame of the student side, and the positive direction of the y-axis is along the height direction of the video frame of the student side; When the upper left corner of the original video frame of the server is used as the origin, the corresponding coordinate system is: starting from the origin, the positive direction of the x-axis is along the width direction of the original video frame of the server, and the positive direction of the y-axis is along the height direction of the original video frame of the server.

5. The AI ​​online education window intelligent adjustment method as described in claim 1, characterized in that: Each request message sent by the student side includes a message ID, the ID of the student side, the IP address of the student side, the second coordinate point of the student side, and a request timestamp.

6. The AI ​​online education window intelligent adjustment method as described in claim 4, characterized in that: When the quantity is less than the preset value, the server calculates the first video frame magnification regions related to the second coordinate points in each request message in sequence, including: The server sets the width of the first video frame magnification region and the second video frame magnification region related to each second coordinate point to be uniformly Area_W, and the height to be uniformly Area_H, and Area_W = a * Srv_W, Area_H = a * Srv_H, where 0 < a < 1, Srv_W is the width of the original video frame of the server, and Srv_H is the height of the original video frame of the server; With the upper left corner of the original video frame of the server as the origin, according to the second coordinate point, the width and height of the first video frame magnification region, and the width and height of the original video frame of the server, calculate the center point coordinates of the first video frame magnification region corresponding to each student side: When the first condition Area_W / 2 <= Srv_Xt <= Srv_W - Area_W / 2 and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2 are satisfied, the first video frame magnification region is completely within the original server frame. At this time, the center point coordinates (Srv_Xo, Srv_Yo) of the first video frame magnification region are the second coordinate point (Srv_Xt, Srv_Yt); When the first condition is not satisfied, the first video frame magnification region is not completely within the original server frame. At this time, it is necessary to adjust the first video frame magnification region so that the region is completely within the original server frame, and the calculation of the center point coordinates of the adjusted first video frame magnification region is as follows: When 0 < Srv_Xt < Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Area_H / 2); When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2 and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_Xt, Area_H / 2); When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W and 0 < Srv_Yt < Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Area_H / 2); When Srv_W - Area_W / 2 < Srv_Xt < Srv_W and Area_H / 2 <= Srv_Yt < Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_H - Area_H / 2, Srv_Yt); When Srv_W - Area_W / 2 <= Srv_Xt < Srv_W and Srv_H - Area_H / 2 <= Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_W - Area_W / 2, Srv_H - Area_H / 2); When Area_W / 2 <= Srv_Xt < Srv_W - Area_W / 2 and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Srv_Xt, Srv_H - Area_H / 2); When 0 < Srv_Xt < Area_W / 2 and Srv_H - Area_H / 2 < Srv_Yt < Srv_H, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_H - Area_H / 2); When 0 < Srv_Xt < Area_W / 2 and Area_H / 2 <= Srv_Yt <= Srv_H - Area_H / 2, the center point coordinates of the enlarged area of the first video frame are (Srv_Xo, Srv_Yo) = (Area_W / 2, Srv_Yt); According to the center point coordinates of the enlarged area of the first video frame and the width and height of the enlarged area of the first video frame, calculate the four-corner coordinates of the enlarged area of the first video frame with the upper left corner of the original video frame of the server as the origin.

7. The AI ​​online education window intelligent adjustment method as described in claim 6, characterized in that: Generate a server enlarged video stream corresponding to the enlarged area of the first video frame, including: According to the four-corner coordinates of the enlarged area of the first video frame with the upper left corner of the original video frame of the server as the origin, extract the pixel data of the corresponding area from the server original video stream, and use an image enlargement algorithm to enlarge the pixel data of the extracted area by 1 / a times proportionally to obtain the server enlarged video stream corresponding to the enlarged area of the first video frame.

8. The AI ​​online education window intelligent adjustment method as described in claim 6, characterized in that: When the quantity is greater than or equal to the preset value, when the server performs regional segmentation on its original video frame, first the server divides its original video frame, divides the original video frame into m equal parts along both the width and the height, then divides the original video frame into m*m squares, and obtains (m + 1)*(m + 1) coordinate points on the original video frame. Among the (m + 1)*(m + 1) coordinate points, the number of coordinate points located on the boundary of the original video frame is 4m, the number of coordinate points located inside the original video frame is (m - 1)*(m - 1), and the coordinate points located inside the original video frame are called the third coordinate points; Set the width of each segmentation area to be uniformly Srv_W’, and the height to be uniformly Srv_H’, and Srv_W’ = b*Srv_W, Srv_H’ = b*Srv_H, where 0 < b < 1, and b > a, b = (m - n) / m, n < m, and n is a positive integer, and there is an overlapping part between each segmentation area, and then obtain multiple segmentation areas from the server original video frame; The server calculates the distances between the points where each second coordinate point is located and the third coordinate points respectively, selects the nearest third coordinate point for the points where each second coordinate point is located, and judges the segmentation area to which each selected third coordinate point belongs, and regards the boundary of the third coordinate point located in the segmentation area as not belonging to the segmentation area, and takes the segmentation area to which it belongs as the segmentation area to which the second coordinate point belongs, and the segmentation area to which it belongs is called the coding area. When there are multiple segmentation areas where the selected third coordinate point is located, randomly select one segmentation area as the coding area, and establish a mapping table between all second coordinate points and the belonging coding areas; Taking the upper left corner of the server's original video frame as the origin, obtain the upper left corner coordinates of each coding area as (Srv_Sca_Xt, Srv_Sca_Yt), and convert the second coordinate point into a fourth coordinate point (Srv_Xt’, Srv_Yt’) = (Srv_Xt - Srv_Sca_Xt, Srv_Yt - Srv_Sca_Yt) with the upper left corner coordinates of the corresponding coding area as the origin; Taking the upper left corner coordinates of the coding area as the origin, calculate the center point coordinates of the enlarged area of the second video frame corresponding to each student terminal according to the fourth coordinate point, the width and height of the enlarged area of the second video frame, and the width and height of the coding area, where the width and height of the coding area are Srv_W’ and Srv_H’ respectively.

9. The AI ​​online education window intelligent adjustment method as described in claim 8, characterized in that: The calculation of the center point coordinates of the enlarged area of the second video frame corresponding to each student terminal includes: When the second condition Area_W / 2 <= Srv_Xt’ <= Srv_W’ - Area_W / 2 and Area_H / 2 <= Srv_Yt’ <= Srv_H’ - Area_H / 2 is satisfied, the enlarged area of the second video frame is completely within the corresponding coding area. At this time, the center point coordinates (Srv_Xo’, Srv_Yo’) of the enlarged area of the second video frame are the fourth coordinate point (Srv_Xt’, Srv_Yt’). When the second condition is not satisfied, the enlarged area of the second video frame is not completely within the corresponding coding area. At this time, it is necessary to adjust the enlarged area of the second video frame so that the area is completely within the corresponding coding area, and the calculation of the center point coordinates of the adjusted enlarged area of the second video frame is as follows: When 0 < Srv_Xt’ < Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Area_H / 2). When Area_W / 2 <= Srv_Xt’ < Srv_W’ - Area_W / 2 and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Area_H / (end]] When Srv_W’ - Area_W / 2 <= Srv_Xt’ < Srv_W’ and 0 < Srv_Yt’ < Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_W’ - Area_W / 2, Area_H / 2). When Srv_W’ - Area_W / 2 < Srv_Xt’ < Srv_W’ and Area_H / 2 <= Srv_Yt’ < Srv_H’ - Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_H’ - Area_H / 2, Srv_Yt’). When Srv_W’ - Area_W / 2 <= Srv_Xt’ < Srv_W’, and Srv_H’ - Area_H / 2 <= Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_W’ - Area_W / 2, Srv_H’ - Area_H / 2); When Area_W / 2 <= Srv_Xt’ < Srv_W’ - Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Srv_Xt’, Srv_H’ - Area_H / 2); When 0 < Srv_Xt’ < Area_W / 2, and Srv_H’ - Area_H / 2 < Srv_Yt’ < Srv_H’, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_H’ - Area_H / 2); When 0 < Srv_Xt’ < Area_W / 2, and Area_H / 2 <= Srv_Yt’ <= Srv_H’ - Area_H / 2, the center point coordinates of the enlarged area of the second video frame are (Srv_Xo’, Srv_Yo’) = (Area_W / 2, Srv_Yt’); Based on the center point coordinates of the enlarged area of the second video frame and the width and height of the enlarged area of the second video frame, calculate the four-corner coordinates of the enlarged area of the second video frame with the upper-left corner of the encoding area as the origin, and convert the four-corner coordinates to the four-corner coordinates with the upper-left corner of the server's original video frame as the origin.

10. The AI ​​online education window intelligent adjustment method as described in claim 9, characterized in that: The generation process of the server enlarged video stream corresponding to the enlarged area of the second video frame includes: The server extracts the pixel data of the corresponding areas of each encoding area from the server's original video stream according to the positions and widths and heights of each encoding area, and uses an image enlargement algorithm to enlarge the pixel data of the extracted area by 1 / b times proportionally to obtain the encoded video stream corresponding to each encoding area; According to the mapping table, find the encoded video stream corresponding to the enlarged area of the second video stream related to each second coordinate point, and send response information including the corresponding encoded video stream and the four-corner coordinates of the enlarged area of the second video frame with the upper-left corner of the server's original video frame as the origin to the corresponding student terminal; After receiving the response information, the student terminal extracts the pixel data of the corresponding area from the corresponding encoded video stream according to the four-corner coordinates of the enlarged area of the second video stream with the upper-left corner of the server's original video frame as the origin to obtain the server enlarged video stream corresponding to the enlarged area of the second video frame.

Images