Method, device, equipment and medium for content interaction based on online course video

Abstract

The present invention relates to a method, device, equipment and medium for content interaction based on online course videos, wherein the method comprises: when the online course video is played, collecting the online course screen, the online course screen is a static screen displayed at a moment when the online course video is played; based on a preset range determination rule, determining the screen range to be analyzed on the online course screen; according to the screen range, intercepting a partial area of ​​the online course screen, and generating a picture of the intercepted partial area; determining the content category of the picture according to the content of the picture, determining the recommended content according to the content category, and displaying the recommended content when the online course video is played. The present invention realizes the autonomous recommendation of extended learning content based on the content of the online course screen, so that students can conveniently and quickly carry out extended learning when watching online courses.

Description

Technical Field

[0001] The present invention relates to the field of teaching interaction technology, and more specifically, to a method, device, equipment and medium for content interaction based on online course videos. Background Art

[0002] When students use devices to watch online courses, they can generally only watch the content of the online course unilaterally, and it is impossible or inconvenient to expand their learning based on the content of the online course. For example, when students watch a course that teaches everyone to recognize fruits, they can only learn the introduction of some fruits in the course, and they cannot continue to expand their learning of other knowledge about these fruits. If they want to continue to expand their learning, they can only search on the search engine themselves. Summary of the invention

[0003] The embodiments of the present invention aim to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a method, apparatus, device and medium for content interaction based on online course videos, so as to solve the problem that the learning content cannot be expanded conveniently and quickly when playing online course videos.

[0004] In a first aspect, an embodiment of the present invention provides a method for content interaction based on an online course video, comprising:

[0005] When the online course video is playing, the online course screen is collected, and the online course screen is a static screen displayed at a moment when the online course video is playing;

[0006] Based on a preset range determination rule, determining a screen range that needs to be analyzed on the online course screen;

[0007] According to the screen range, a partial area of ​​the online course screen is intercepted, and a picture is generated from the intercepted partial area;

[0008] Determine the content category of the picture according to the content of the picture;

[0009] Determine recommended content according to the content category, and display the recommended content when the online course video is played.

[0010] Optionally, based on a preset range determination rule, determining a screen range to be analyzed on the online course screen includes:

[0011] Based on a plurality of preset correspondences between video characteristics and screen ranges, the screen range that needs to be analyzed on the online course screen is determined according to the video characteristics of the online course video.

[0012] Optionally, the video characteristic is an online course type or an online course screen layout;

[0013] Optionally, based on a preset range determination rule, determining a screen range to be analyzed on the online course screen includes:

[0014] Obtain the analysis scope circled by the user on the online course screen;

[0015] Based on a preset range determination rule, according to the analysis range, a screen range that needs to be analyzed is determined on the online course screen.

[0016] Optionally, based on a preset range determination rule and according to the analysis range, determining a screen range to be analyzed on the online course screen includes:

[0017] Based on the preset shape, the smallest screen having the preset shape and including the analysis range is used as the screen range to be analyzed in the online course screen.

[0018] Optionally, when the online course video is played, the online course screen is collected, including:

[0019] When the online course video is played, the online course screen is captured multiple times at a fixed rate, and each time, one of the online course screens displayed at the current moment is captured.

[0020] Optionally, determining the content category of the picture according to the content of the picture includes:

[0021] For one of the online course screens, one of the screen ranges corresponds to capturing a partial area of ​​the online course screen to generate a picture;

[0022] If the currently generated picture is the first picture generated corresponding to the picture range, determining the content category of the currently generated picture according to the content of the currently generated picture;

[0023] If the currently generated picture is not the first picture generated corresponding to the picture range, a difference detection is performed between the currently generated picture and the picture corresponding to the same picture range generated last time to obtain the degree of difference;

[0024] According to the size of the difference degree, it is determined whether to determine the content category of the currently generated picture according to the content of the currently generated picture.

[0025] Optionally, determining recommended content according to the content category, and displaying the recommended content when the online course video is played, includes:

[0026] For the online course screen collected this time, if the content category of the generated picture is determined according to the content of the generated picture, the recommended content corresponding to the online course screen is determined according to the content category, and the recommended content is displayed when the online course video is played;

[0027] If the content category of the generated image is not determined based on the content of the generated image, the recommended content corresponding to the online course screen collected before this time will be displayed when the online course video is played.

[0028] Optionally, determining the content category of the picture according to the content of the picture includes:

[0029] Input the image into a multi-label classification model, determine the possibility of the image containing each content category according to the content of the image, and output the probability of whether the image contains each content category; when the probability is greater than a preset probability threshold, use the content category corresponding to the probability as the content category of the image;

[0030] Alternatively, the image is input into a multi-classification model, the possibility of the image belonging to each content category is determined according to the content of the image, the probability of the image belonging to each content category is output, and the content category corresponding to the maximum probability is used as the content category of the image.

[0031] Optionally, determining the recommended content according to the content category includes:

[0032] Determine a recommendation category according to the content category, wherein the recommendation category includes one or more elements;

[0033] One or more elements are selected from the recommendation category as recommended content.

[0034] Optionally, selecting one or more elements from the recommendation category as recommended content includes:

[0035] Extracting feature information of the element and the picture respectively, wherein the feature information includes one or more of the following: color and shape;

[0036] One or more of the elements are selected from the recommendation category as recommended content according to a matching degree between the feature information of the element and the picture.

[0037] In a second aspect, an embodiment of the present invention provides a device for content interaction based on online course videos, including:

[0038] The picture acquisition module is used to collect the online class picture when the online class video is played, and the online class picture is a static picture displayed at a certain moment when the online class video is played;

[0039] A range determination module, used to determine the screen range that needs to be analyzed in the online course screen based on a preset range determination rule;

[0040] A picture generation module, used to capture a partial area of ​​the online course screen according to the screen range, and generate a picture of the captured partial area;

[0041] The recommendation display module is used to determine the content category of the picture according to the content of the picture, determine the recommended content according to the content category, and display the recommended content when the online course video is played.

[0042] In a third aspect, an embodiment of the present invention provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the above-mentioned method for content interaction based on online course videos is implemented.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned method for content interaction based on online course videos.

[0044] Compared with the prior art, the beneficial effects of the embodiments of the present invention are as follows: the embodiments of the present invention collect online course screens when the online course video is played, generate pictures that need to be analyzed based on the online course screens, analyze the contents of the pictures to obtain recommended content, and display the recommended content when the online course video is played, thereby realizing the autonomous recommendation of extended learning content based on the content of the online course screen, so that students can carry out extended learning conveniently and quickly while watching online courses, and when generating pictures that need to be analyzed, partial areas of the online course screen are captured based on the screen range, thereby reducing the screen content that needs to be analyzed, making the analysis more targeted and accurate, and making the recommended extended learning content more in line with students' needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a method for content interaction based on online course videos according to Embodiment 1 of the present invention.

[0046] Figure 2 This is a flow chart of a method for content interaction based on online course videos according to Embodiment 2 of the present invention.

[0047] Figure 3 This is a diagram showing the composition of an apparatus for content interaction based on online course videos according to Embodiment 3 of the present invention.

[0048] Figure 4 This is another composition diagram of an apparatus for content interaction based on online course videos according to Embodiment 3 of the present invention.

[0049] Figure 5 This is a diagram of the computer device composition of Example 3 of the present invention. DETAILED DESCRIPTION

[0050] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0051] Example 1

[0052] This embodiment provides a method for content interaction based on online course videos, which can be applied to a device that is playing an online course video, or a control device that controls a device that is playing an online course video, and can also be applied to a device that is not playing an online course video or even does not have the function of playing an online course video, but can cooperate with a device that is playing an online course video. Online course videos refer to videos that allow students to watch and learn knowledge from them, which can be pre-recorded or live broadcast. Online course videos usually feature lecturers and some things displayed by the lecturers, such as display courseware, teaching aids, etc. These things reflect the content that students are learning. Identifying these things can display more learning materials related to these things, allowing students to further expand their learning.

[0053] like Figure 1 As shown, the method for content interaction based on online course videos provided in this embodiment can identify things that appear in online course videos and timely recommend content that can be expanded for learning based on the identification, which includes the following steps:

[0054] S101. When the online course video is playing, collect the online course screen.

[0055] The online class screen is a static screen displayed at a certain moment when the online class video is played. The online class video is played dynamically. In order to accurately analyze and identify the objects that appear in the dynamic online class video, a static online class screen can be obtained by collecting the online class screen.

[0056] Specifically, when the online course video is playing, the online course screen that is currently being played can be captured as the collected online course screen by means of screenshots; when the online course video is playing, data of a frame of the online course video at the current moment can be extracted from the data stream of the online course video, and processed into a frame of the online course screen according to the extracted data. The collected online course screen can be stored first.

[0057] S102. Based on a preset range determination rule, determine the screen range that needs to be analyzed in the online course screen.

[0058] The online class screen is collected from the online class video being played. In addition to the things that reflect the students' learning content, the online class video will also show other contents, such as the face of the lecturer, the container for loading things, etc. These other contents will interfere with the analysis accuracy of the things that reflect the students' learning content, and even make it impossible to analyze and identify the things. Therefore, it is necessary to determine the screen range that needs to be analyzed based on the position of the things that reflect the learning content on the online class screen to improve the accuracy of the analysis.

[0059] The scope determination rule is a pre-set rule for determining the scope of the screen that needs to be analyzed in the online course screen. It can be pre-set manually by a developer, a student watching the online course video, or a parent or teacher who guides the student to watch the online course video.

[0060] The screen range is the pixel area information or coordinate area information indicating a certain area within the online course screen. For example, for an online course screen of 400 pixels × 600 pixels, the screen range can be a pixel area with a length of x pixels and a width of y pixels, with a pixel point (X, Y) in the pixel coordinate system as the midpoint. This pixel area is a part of the online course screen. For another example, for an online course screen of 9cm × 16cm, the screen range can be a rectangular area formed by 4 points (x1, y1), (x2, y2), (x3, y3), (x4, y4) in the length coordinate system. This rectangular area is a part of the online course screen.

[0061] Based on the preset range determination rules, the screen range that needs to be analyzed on an online course screen may be one or more. The multiple screen ranges determined on the same online course screen do not affect each other and may overlap or intersect with each other.

[0062] Determining the scope of the screen that needs to be analyzed can narrow the content to be analyzed from the entire online course screen to the determined screen range, making the analysis more targeted. The final recommendations can more accurately grasp user needs, improve user experience, and reduce the analysis workload accordingly, thereby improving the efficiency of analysis and recommendation.

[0063] There are many ways to implement how to determine the screen range that needs to be analyzed in the online course screen based on the preset range determination rules.

[0064] In one implementation, the preset range determination rule may be a correspondence between a plurality of preset video characteristics and a screen range. Accordingly, based on the preset range determination rule, determining the screen range to be analyzed on the online course screen includes: determining the screen range to be analyzed on the online course screen according to the video characteristics of the online course video based on the correspondence between a plurality of preset video characteristics and a screen range.

[0065] Video characteristics are the unique properties of online course videos. Several video characteristics can be pre-set based on their impact on the determination of the screen range, and the correspondence between these several video characteristics and the screen range can be pre-set. When the video characteristics of the online course video are determined, the screen range corresponding to the video characteristics can be determined based on this correspondence, that is, the screen range that needs to be analyzed in the online course screen is determined. It can be understood that each video characteristic corresponds to at least one screen range.

[0066] Through the preset correspondence between video characteristics and screen range, it is convenient to quickly determine the screen range that needs to be analyzed for the online course video based on the video characteristics of different online course videos, and it can adapt to the unique properties of different online course videos and determine the screen range more specifically, so that the partial area of ​​the online course screen that is subsequently captured based on the screen range can more accurately locate the things that reflect the students' learning content, thereby making the recommendation more accurate.

[0067] The video characteristics may be online course type or online course screen layout, etc. The online course type refers to the category to which the online course video belongs. The category may include several course subjects or several course themes, such as Chinese, English, test question explanation topics, review topics, etc. The online course screen layout refers to the arrangement and planning of each element displayed on the online course screen, such as the position and size of each element on the online course screen.

[0068] The type of online course and the layout of the online course screen are relatively obvious characteristics of online course videos, which can make it easier and more accurate to determine the video characteristics of online course videos. The type of online course and the layout of the online course screen have a great influence on the part of the online course screen where the student learning content appears. As video characteristics, they correspond to the range determination rules, which is more conducive to the accurate determination of the screen range.

[0069] For courses of the same type (such as the same course subject or the same course theme, etc.), during the online course video playback, the objects that need to be identified and analyzed will basically appear in a fixed size and fixed position on the online course screen. Therefore, when the video feature is the online course type, step S102 can be specifically as follows: pre-set a number of course types corresponding to the screen range that needs to be analyzed on the online course screen, and determine the screen range that needs to be analyzed on the online course screen according to the course type to which the online course video belongs.

[0070] For example, a course with the theme of explaining test questions, that is, the online course video belongs to the course type of explaining test questions. During the playback of the online course video, the online course screen will often be displayed in the form of questions on the left half and answers on the right half. The screen range that needs to be analyzed for the course on this theme can be pre-set, which can be the upper-middle left half of the online course screen and occupies 2 / 3 of the left half of the online course screen size, and the upper-middle right half of the screen and occupies 2 / 3 of the right half of the online course screen size.

[0071] For the layout of the online course screen, on the one hand, the layout of the online course screen includes the overall layout of the online course screen, and the overall layout is composed of various elements. For example, the online course video generally displays the courseware at a set position and a set size, and then displays the state expression of the lecturer at another set position and a set size. The courseware, the state expression of the lecturer, etc. are the elements that constitute the overall layout. On the other hand, the layout of the online course screen also includes the internal layout of these elements, such as the internal layout of the courseware. For the courseware made by the same lecturer, or the courseware of the same type of course, the layout of the courseware display is similar, and the things that need to be identified often appear in a fixed size and fixed position in some specific positions of the courseware screen. Therefore, when the video feature is the layout of the online course screen, step S102 can be specifically as follows: pre-set a number of online course screen layouts corresponding to the screen range that needs to be analyzed on the online course screen, and determine the screen range that needs to be analyzed on the online course screen according to the online course screen layout to which the online course video belongs.

[0072] For example, the online course screen layout of a certain online course video is: the overall screen layout of the online course screen is that the courseware is displayed on the entire online course screen, the status expression of the teacher is displayed in the corner, and the internal screen layout of the courseware is to place the things that students need to learn in the middle. Then, such an online course screen layout can be pre-set so that the screen range that needs to be analyzed is the middle of the online course screen and occupies 1 / 2 of the size of the entire online course screen.

[0073] It can be understood that the pre-set several online course screen layouts correspond to the screen range that needs to be analyzed in the online course screen. It can be the correspondence of several overall screen layouts, the correspondence of several element internal screen layouts, or the correspondence of several overall screen layouts and several element internal screen layouts.

[0074] It is also understandable that for different types of online courses or different online course screen layouts, the same screen range can be uniformly preset.

[0075] In another embodiment, the screen range that needs to be analyzed on the online course screen can be determined based on the analysis range circled by the user. The user can be the student watching the online course video, or a parent or teacher who guides the student to watch the online course video. This can not only enhance the interactivity between the user and the online course video during playback, but also make subsequent recommendations more in line with the user's extended learning needs. At this time, the preset range determination rule is specifically a rule for determining the screen range that needs to be analyzed on the online course screen based on the analysis range circled by the user, that is, the preset range determination rule is a rule for changing the screen range from the analysis range. Therefore, step 102 can be: obtaining the analysis range circled by the user on the online course screen; based on the preset range determination rule, determine the screen range that needs to be analyzed on the online course screen according to the analysis range.

[0076] The shapes of the analysis ranges circled by users are various. Based on the preferences of different users or different preset circle rules, they may use rectangular frames or circular frames for selection, or they may draw irregular tracks by hand. The shapes of the screen ranges are not uniform, which will make the shapes of the pictures generated by subsequently capturing part of the online course screen inconsistent, which is not conducive to determining the content category of the picture in a unified way. Therefore, the preset range determination rules can be preset shapes. No matter what shape the analysis range circled by the user is, the determined screen range can be unified into the preset shape. On the basis of the unified shape, it is also necessary to consider whether the determined screen range covers the analysis range circled by the user, to prevent some of the content circled by the user from being captured and generated into pictures because it is not covered within the screen range, resulting in the recommendation not fully meeting the needs of the user.

[0077] Accordingly, step S102 may be more specifically as follows: obtaining the analysis range circled by the user on the online course screen, and based on a preset shape, taking the smallest screen on the online course screen with a preset shape and including the analysis range as the screen range to be analyzed.

[0078] For example, the preset shape is a rectangle, and the analysis range circled by the user on the online course screen is a circular range, then a minimum rectangular screen range including the circular range is selected on the online course screen, and the circular range circled by the user is the inscribed circle of the minimum rectangular screen range. The minimum rectangular screen range is used as the screen range that needs to be analyzed, which can appropriately expand the analysis range circled by the user to ensure that the content circled by the user is covered within the screen range.

[0079] It is understandable that in addition to presetting the shape and standardizing the shape of the screen range, you can also preset the size and standardize the size of the screen range, so that the size of the pictures generated by capturing the online course screen according to the screen range can be unified.

[0080] S103. According to the screen range, capture a partial area of ​​the online course screen, and generate a picture of the captured partial area.

[0081] After the online course screen determines the screen range that needs to be analyzed, a partial area of ​​the online course screen within the screen range can be captured according to the determined screen range, and a picture is generated for the extracted partial area. Specifically, after the screen range that needs to be analyzed is determined, the pixel area information or coordinate area information of a certain partial area in the online course screen indicated by the screen range that needs to be analyzed is also determined, and the corresponding partial area is captured in the online course screen according to the pixel area information or coordinate area information, and a picture is generated for the captured partial area.

[0082] S104. Determine the content category of the picture according to the content of the picture, determine recommended content according to the content category, and display the recommended content when the online course video is played.

[0083] After capturing the online course screen and generating an image, the content of the image can be analyzed to obtain the content category to determine the recommended content that needs to be displayed when the online course video is played. The content of the image refers to the various valid information contained in the image. The recommended content refers to various information or links to such information recommended to students who are watching the online course video.

[0084] After the image is generated and before the content category of the image is determined according to the content of the image, some pre-processing may be performed on the image. The purpose of the image pre-processing may be to unify the size, brightness, etc. of the image so that the feature information of the image can be analyzed uniformly later.

[0085] For the unified image size, taking the screen range as a rectangle as an example, according to the rectangular screen range, the extracted part of the online course screen is a rectangle, and the extracted part of the rectangle is transformed to a specified size M×N, and the specified size can be a specified resolution (the units of M and N are pixels at this time), or a specified physical size (the units of M and N are length units at this time). The size transformation method can be proportional transformation, and the insufficient part after the transformation is filled with 0 or 1 pixel. The advantage of using this size transformation method is that the content of the picture will not be deformed, but there are other methods for picture size transformation, such as stretching, scaling, cropping, etc. The proportional transformation and filling the insufficient part with 0 or 1 pixel size transformation method is only an example, and this embodiment does not limit this.

[0086] The content category of the picture is determined according to the content of the picture, and different classification methods can be used specifically.

[0087] One of the classification methods is to input the image into a multi-label classification model, determine the possibility of whether the image contains each content category based on the content of the image, and output the probability of whether the image contains each content category. When the probability is greater than a preset probability threshold, it is determined that the image contains the content category corresponding to the probability, and the content category corresponding to the probability is used as the content category of the image.

[0088] The multi-label classification model can use a conventional network model, such as the typical residual neural network resnet50, resnet101, etc., or a network model formed by a combination of different deep learning blocks. The multi-label classification loss function is constructed using the idea of ​​binary relevance, and a trained multi-label classification model can be obtained by training with labeled sample data.

[0089] The idea of ​​Binary Relevance is to decompose the multi-label learning problem into several independent binary classification problems, each of which corresponds to a content category in the category space. For a sample x, Binary Relevance calculates each content category y by using each binary classifier j The category correlation of predicting the category set of sample x:

[0090] Y={y j |g j (x)>0,1≤j≤q}

[0091] Specifically, the content category y of the multi-label classification model is pre-set j , there are q content categories in total, j = 1, 2, ..., q. In the multi-label classification model, q independent binary classifiers g are formed. j (x), the multi-label classification model is trained by annotating sample data with the content category. After the image is input into the trained multi-label classification model, each binary classifier g j (x) can predict whether the image contains content category y j The possibility of the picture belonging to content category y j The probability {p j |1≤j≤q}, probability p j This means that the image contains content category y j probability.

[0092] The probability threshold p can be preset 阈值 , if the probability p j Greater than the preset probability threshold p 阈值 , then the input image is considered to contain content category y j , if there does not exist a probability p jGreater than the preset probability threshold p 阈值 , then the input image is considered to not contain any of the pre-set content categories y j .

[0093] Taking the online course video of the course on cognition of life things as an example, three content categories of the multi-label classification model can be pre-set: fruit, transportation, and animals; three independent binary classifiers are formed in the multi-label classification model: fruit binary classifier, transportation binary classifier, and animal binary classifier.

[0094] After the image is input into the trained multi-label classification model, the fruit classifier will predict whether the image contains fruit and obtain the probability p of containing fruit. 水果 , the probability of not containing fruit is 1-p 水果 The vehicle class binary classifier will predict whether the image contains a vehicle and obtain the probability p of containing a vehicle. 交通工具 , the probability of not containing transportation is 1-p 交通工具 ; The animal class binary classifier will predict whether the image contains an animal and obtain the probability p of containing an animal 动物 , the probability of not containing animals is 1-p 动物 .

[0095] Preset probability threshold p 阈值 =80%. Assuming probability p 水果 =85%, probability p 交通工具 =90%, probability p 动物 =13%, then p 水果 >p 阈值 、p 交通工具 >p 阈值 , we can assume that the input image contains fruits and vehicles, and we can determine that the content categories of the image are fruits and vehicles. Assume that the probability p 水果 =60%, probability p 交通工具 =55%, probability p 动物 =21%, then p 水果 、p 交通工具 、p 动物 No more than p 阈值 , it can be considered that the input image does not contain fruits, vehicles and animals. At this time, it can be determined that the content category of the image is empty.

[0096] When the content category of the image is empty, the recommended content determined according to the image content category may also be empty, and empty recommended content may be displayed when the online course video is played, or no recommended content may be displayed.

[0097] Since online course images usually contain one or more content categories, a multi-label classification model can more completely obtain the content categories contained in the image compared to other conventional classification models, and can also handle the situation where the input image does not contain any content category.

[0098] Another classification method is to input the image into a multi-classification model, determine the possibility of the image belonging to each content category based on the content of the image, output the probability that the image belongs to each content category, and use the content category corresponding to the maximum probability as the content category of the image.

[0099] The multi-classification model can use a conventional network model, such as the typical residual neural network resnet50, resnet101, etc., or a network model formed by combining different deep learning blocks. By training with labeled sample data, a trained multi-label classification model can be obtained.

[0100] Specifically, there are q-1 content categories y pre-set. j , j = 1, 2, ..., q-1, the qth content category does not belong to q-1 content categories y j The multi-label classification model is trained by using sample data labeled with the content category. By inputting the image into the trained multi-label classification model, we can get the image belonging to each content category y j and The probability {p j |1≤j≤q}, according to the probability p j The relative size of the image determines whether it belongs to q-1 content categories y j Which one or several of them, or whether it does not belong to q-1 content categories y j Any one of them. With probability p q When it is not the maximum, select probability p j The largest one or several corresponding content categories y j as the content category of the picture; with probability p q When the maximum value is reached, the content category of the image does not belong to q-1 content categories y j Any one of .

[0101] Taking the online course video of the course on cognition of daily life as an example, four content categories of the multi-classification model can be pre-set: fruits, transportation, animals, and other categories (not belonging to any of the fruit, transportation, and animal categories).

[0102] After the image belongs to the trained multi-classification model, we can get the probability p that the image belongs to fruit, transportation, animal, and other categories respectively. 水果 、p交通工具 、p 动物 、p 其他 , where p 水果 +p 交通工具 +p 动物 +p 其他 =1, then according to p 水果 、p 交通工具 、p 动物 、p 其他 The relative sizes of these four determine the content category of the image.

[0103] Assume probability p 水果 =56%, p 交通工具 =34%, p 动物 =7%, p 其他 =3%, the largest is p 水果 , the next largest is p 交通工具 , we can determine that the content category of the picture is fruit or fruit and transportation. Assume that the probability p 水果 =10%, p 交通工具 =8%, p 动物 =22%, p 其他 = 60%, the largest is p 其他 , it can be determined that the content category of the image is other categories.

[0104] Similarly, when the content category of the image is other categories, the recommended content determined according to the image content category may be empty, and empty recommended content may be displayed when the online course video is played, or no recommended content may be displayed.

[0105] The use of multi-classification models can more accurately determine the content category of the image. With the same amount of training data, it generally has a higher classification accuracy and can also handle situations where the input image does not belong to any content category. The use of multi-classification models is suitable for the need to recommend only one content category from a picture generated from an online class screen.

[0106] After obtaining the content category of the image, a two-step recommendation strategy can be adopted: first determine the recommended category, and then determine the recommended content, thereby improving the accuracy of the recommendation.

[0107] Determining recommended content according to the content category specifically includes: determining a recommended category according to the content category, the recommended category including one or more elements; and selecting one or more elements from the recommended category as recommended content.

[0108] More specifically, determining the recommended category based on the content category may be: when the output result of the multi-label classification model is that the image contains one or more content categories, determining the recommended category to be all or part of the content categories contained in the image; when the recommended category is only part of the content categories contained in the image, it is necessary to filter the content categories contained in the image, and the filtering may be performed based on the probability that the image contains the content category, or based on other factors, which is not limited in this embodiment. When the output result of the multi-label classification model is that the image does not contain any content category, determining that the recommended content is empty and not recommending it.

[0109] In the specific implementation process, when building a multi-label classification model, the content category can be set according to the recommendation category. In some practical scenarios, the content category can be in a one-to-one correspondence with the recommendation category, that is, the content category and the recommendation category are the same.

[0110] There are many implementations for selecting one or more elements from a recommendation category as recommended content.

[0111] In one implementation, one or more elements are randomly selected from the recommendation category as the recommended content. If there are multiple recommendation categories, one element can be randomly selected from each recommendation category as the recommended content.

[0112] In another implementation, feature information of elements and images may be extracted respectively, and the feature information may include one or more of the following: color, shape; and based on the matching degree of the feature information of elements and images, one or more elements may be selected from the recommendation category as recommended content.

[0113] It can be understood that in this embodiment, the element exists in the data format of a picture, or in a data format that can be converted into a picture, and feature information can be extracted therefrom and matched with the feature information of the picture.

[0114] As described above, the characteristic information of a picture refers to characteristic information that can represent the content of the picture, such as color, shape, or a combination of the two. Correspondingly, the characteristic information of an element also refers to characteristic information that can represent the content of the element.

[0115] Among them, color may include the types of colors contained, the number of color types, etc., and shape may include the types of shapes contained, the number of shape types, etc.

[0116] To extract the feature information of an element or a picture, the following methods can be used: extract the RGB information of each pixel of the element or picture to obtain the color of the element or picture; use the edge detection algorithm to extract the outline of each object in the picture. The principle of the edge detection algorithm is to regard non-continuous pixels with large changes as edges. OpenCV (an open source computer vision and machine learning software library) provides many image edge detection functions, such as Laplacian(), Sobel(), Scharr(), etc.

[0117] The matching degree between the characteristic information of the element and the characteristic information of the image can be determined in the following way:

[0118] Train a prediction network for feature information, input elements and pictures into the trained prediction network, and predict the feature information of elements and pictures respectively. Feature information can be recorded in the form of information matrix, thus forming a feature information matrix for recommending category j element i The feature information matrix v of the image x , according to the following formula, calculate the two feature information matrices and v x Similarity:

[0119]

[0120] In the above formula, i=1, 2, ..., m, where m is the total number of elements included in the recommended category j. The larger the value, the higher the matching degree between the feature information of the element and the feature information of the image. The largest element or elements to recommend.

[0121] Determining recommended content based on the matching degree of feature information between elements and images can make the recommended content more relevant to the image, make the recommendation more accurate, and improve the user experience. For example, if the image shows an orange, then when selecting recommended elements in the recommended category of fruit, it will be more inclined to recommend tangerines rather than bananas in the fruit category, because oranges and tangerines have more similar colors and shapes than oranges and bananas.

[0122] The recommended content is displayed when the online course video is playing. Specifically, the recommended content can be displayed in a fixed position of the online course video in a message box in a random order. The recommended content corresponding to each recommendation category can be displayed for a fixed time, and different recommendation categories can be displayed in a loop. When a recommended content is selected by the user, the corresponding content link can be adjusted to display the detailed information of the recommended content; when returning to continue playing the online course video, the remaining recommended content that has not been selected by the user will continue to be displayed.

[0123] Example 2

[0124] Based on the same inventive concept as in Example 1, Figure 2 As shown, this embodiment provides a method for content interaction based on online course videos, which is a further optimization of Embodiment 1, and specifically includes:

[0125] S201. When the online course video is playing, the online course screen is captured multiple times at a fixed rate, and each time an online course screen displayed at the current moment is captured.

[0126] Specifically, along the online course video playback timeline, the online course screen is collected multiple times at a fixed rate. When an online course video is played, the collection rate of the online course screen is fixed. Taking a 20-minute online course video as an example, along the online course video playback timeline, the online course screen can be collected multiple times at a fixed rate of 5 minutes / time, and the online course screen will be collected 4 times on the entire online course video playback timeline.

[0127] The fixed rate for capturing online course images can be a fixed value preset manually, or a fixed value determined according to a preset rate rule. The rate rule can be a rule related to the duration of the online course video. For example, the fixed rate is negatively correlated with the duration of the online course video. For every 5 minutes increase in duration, the value of the fixed rate increases by 1 minute / time.

[0128] When presetting a fixed rate, the subsequent steps based on the collected online course images, the requirements for device processing efficiency, and the limitations of device hardware performance can be considered.

[0129] The collected multiple online course screens can be stored in a temporary space in sequence, in a picture format or a format convertible into a picture, so that the online course screens can be conveniently read in sequence from the temporary space for subsequent picture generation.

[0130] The online class screen is collected at a fixed rate, and each time an online class screen is collected, an image is generated and content is recommended for the collected online class screen. The things that can be analyzed and identified can be captured from the online class video in real time and continuously, so as to perform real-time and continuous content recommendation.

[0131] S202. Based on a preset range determination rule, determine the screen range that needs to be analyzed in the online course screen.

[0132] The scope determination rule is a pre-set rule for determining the scope of the screen that needs to be analyzed in the online course screen. It can be pre-set manually by a developer, a student watching the online course video, or a parent or teacher who guides the student to watch the online course video.

[0133] Determining the scope of the screen that needs to be analyzed can not only narrow down the content that needs to be analyzed from the entire online course screen to the determined screen range, but also make the analysis more targeted. The recommended content finally displayed can more accurately grasp the user's needs. In addition, it can also reduce the amount of calculation when the online course screen is collected multiple times, and improve the calculation rate of analysis and recommendation, thereby increasing the collection rate of the online course screen and improving the accuracy of analysis and recommendation.

[0134] S203. According to the screen range, a partial area of ​​the online course screen is captured, and a picture is generated from the captured partial area. For one online course screen, one screen range corresponds to a partial area of ​​the online course screen captured to generate a picture.

[0135] Each time an online course screen is collected, step S203 is executed once, and a partial area of ​​the online course screen is captured according to a determined screen range, and the captured partial area is generated into a picture, that is, one online course screen and one screen range will be processed accordingly to generate one picture.

[0136] One online course screen can determine several screen ranges that need to be analyzed, that is, one online course screen can be processed to generate multiple pictures, and different pictures generated by one online course screen correspond to different screen ranges.

[0137] S204. Determine whether the currently generated picture is the first picture generated corresponding to a screen range. If so, execute steps S208 to S212; if not, execute steps S205 to S206.

[0138] For each of the different online course screens collected, the pictures generated corresponding to each screen range execute step S204 once to determine whether the generated picture is the first picture generated corresponding to the screen range. The first picture generated corresponding to the screen range refers to the picture generated for the first time corresponding to the screen range, that is, based on the screen range, a partial area of ​​the online course screen collected for the first time is intercepted, and the first picture is generated based on the intercepted partial area.

[0139] S205. Perform a difference detection on the currently generated picture and the last generated picture corresponding to the same picture range to obtain the degree of difference.

[0140] When it is determined that the currently generated picture is not the first picture generated corresponding to a certain screen range, then the currently generated picture is not the first picture generated corresponding to the screen range, and there is a picture generated last time corresponding to the screen range. The currently generated picture and the picture generated last time corresponding to the screen range are detected for difference to obtain the degree of difference. Because the currently generated picture is not the first picture generated corresponding to the screen range, the content analysis and subsequent recommended content determination have been performed on the pictures corresponding to the screen range before, so the degree of difference between the previously generated pictures can be detected first. When the degree of difference is small, the content category of the currently generated picture is not determined. When the degree of difference is large, the content category of the currently generated picture is determined, thereby avoiding unnecessary duplication of work and wasting computing resources.

[0141] In one implementation, the degree of difference may be calculated in the following manner: extract information from the currently generated image and the last generated image to form information matrices Img1 and Img2, respectively, and calculate the degree of difference ε using the following formula:

[0142] ε = Mean (Img1 - Img2);

[0143] In the above formula, Mean() is a function that calculates the average value of each element in the matrix.

[0144] In another implementation, the degree of difference may also be judged according to the size, position, number, etc. of the regions that are different between the currently generated image and the last generated image.

[0145] The degree of difference can be judged quantitatively or qualitatively.

[0146] Of course, similarly, the currently generated image can also be tested for similarity with the last generated image to obtain the degree of similarity. When the degree of similarity is large, the content category of the currently generated image is not determined. When the degree of similarity is small, the content category of the currently generated image is determined. Both the degree of similarity and the degree of difference can be used to judge the similarity of the two generated images.

[0147] S206. Determine whether to determine the content category of the currently generated image based on the content of the currently generated image according to the degree of difference. If so, execute steps S208 to S211. If not, execute step S207.

[0148] A condition regarding the degree of difference may be set to determine whether to determine the content category of the currently generated picture based on the content of the currently generated picture.

[0149] When the degree of difference is a quantitative judgment, a preferred implementation is to preset a difference threshold, determine whether the degree of difference is less than the preset difference threshold, and if so, execute steps S208 to S212, and if not, execute step S207.

[0150] When the degree of difference is a qualitative judgment, a preferred implementation is to preset a judgment index, and to execute steps S208 to S212 when the judgment index is reached, and to execute step S207 when the judgment index is reached.

[0151] S207. Display the recommended content corresponding to the previously collected online course images when the online course video is played.

[0152] Step S207 is performed when the difference between the currently generated picture and the last generated picture corresponding to the same screen range is small, that is, there is a picture generated last time corresponding to the screen range, which also means that the online class screen collected this time is not collected for the first time, that is, there is an online class screen collected before this time. When the content category is not determined based on the content of the currently generated picture, the recommended content corresponding to the previously collected online class screen can be displayed when the online class video is played.

[0153] It is understandable that the online class screen collected before this time may be the last time. When the content category of the online class screen collected last time is not determined according to the content of the generated picture, the online class screen collected before this time may be several times.

[0154] Directly using the recommended content corresponding to the previously collected online course images for display can ensure that during the online course video playback, the recommended content display will not be interrupted due to the small difference between the images generated corresponding to the two collected online course images, and appropriate recommended content can be continuously provided. In addition, reusing the previously determined recommended content can reduce the time for determining the recommended content, speed up the recommendation, and improve the experience fluency of the recommendation service.

[0155] It is understandable that in order to reuse the previously determined recommended content, the previously determined recommended content may be cached.

[0156] S208. Determine the content category of the currently generated picture according to the content of the currently generated picture.

[0157] When it is determined that the currently generated picture is the first picture generated corresponding to a certain screen range, step S208 can be executed to determine the content category of the currently generated picture according to the content of the currently generated picture. Because the currently generated picture is the first picture generated corresponding to the screen range, and no content analysis and subsequent determination of recommended content have been performed on the pictures corresponding to the screen range, it is necessary to determine the content category of the currently generated picture according to the content of the currently generated picture.

[0158] When it is determined that the difference between the current and last generated pictures corresponding to the same screen range is large, step S208 may also be executed to determine the content category of the current generated picture according to the content of the current generated picture. Because the two pictures are very different, the previously determined recommended content cannot be reused, so it is necessary to perform content analysis on the current generated picture and determine the subsequent recommended content, and determine the content category of the current generated picture according to the content of the current generated picture.

[0159] Specifically, step S208 may include:

[0160] Input the currently generated image into the multi-label classification model, determine the possibility of whether the image contains each content category, output the probability of whether the image contains each content category, and when the probability is greater than the preset probability threshold, use the content category corresponding to the probability as the content category of the currently generated image; or input the currently generated image into the multi-classification model, determine the possibility of the image belonging to each content category, output the probability of the image belonging to each content category, and use the content category corresponding to the maximum probability as the content category of the currently generated image.

[0161] S209. Determine a recommendation category according to the content category, where the recommendation category includes one or more elements.

[0162] Before determining the recommended content to be displayed, first determine the recommendation category. The recommendation category is formed by classifying the recommended content, which can make the final recommended content more accurate.

[0163] A recommendation category can have only one element or multiple elements.

[0164] S210. Extract feature information of elements and images respectively, where the feature information includes one or more of the following: color and shape.

[0165] After determining the recommendation category, it is necessary to determine which element in the recommendation category is used as the final recommendation content. The feature information of the elements and images extracted separately can be used to more easily and accurately match the elements and images.

[0166] S211. Select one or more elements from the recommendation category as recommended content according to the matching degree between the feature information of the element and the image.

[0167] Specifically, all elements whose matching degree meets the preset matching condition may be selected from the recommendation category as the recommended content, or the element whose matching degree best meets the preset matching condition may be selected as the recommended content.

[0168] The matching degree can be obtained by comprehensive evaluation based on the matching conditions between colors and the matching conditions between shapes. The preset matching condition can be a preset matching degree threshold, a matching degree range, or a plurality of logical conditions regarding the matching conditions of colors and shapes.

[0169] S212. Display the recommended content when the online course video is playing.

[0170] The determined recommended content will be displayed while the online course video is playing, realizing real-time recommendation, so that users can see the recommended content at any time while the online course video is playing, and can even directly enter the corresponding link of the recommended content through the displayed recommended content.

[0171] Example 3

[0172] Based on the same inventive concept as in Example 1 and Example 2, Figure 3 As shown, this embodiment provides a device for content interaction based on online course videos, which can be applied to a device that is playing an online course video, or a control device that controls a device that is playing an online course video, and can also be applied to a device that is not playing an online course video or even has no online course video playing function, but can cooperate with a device that is playing an online course video, and includes:

[0173] The picture acquisition module 301 is used to acquire the current online class picture when the online class video is played. The online class picture is a static picture displayed at a certain moment when the online class video is played;

[0174] A range determination module 302 is used to determine the screen range that needs to be analyzed in the online course screen based on a preset range determination rule;

[0175] The picture generation module 303 is used to capture a part of the online course screen according to the screen range, and generate a picture from the captured part of the screen;

[0176] The recommendation display module 304 is used to determine the content category of the picture according to the content of the picture, determine the recommended content according to the content category, and display the recommended content when the online course video is played.

[0177] There are many ways to implement how the range determination module 302 determines the screen range that needs to be analyzed on the online course screen based on the preset range determination rule.

[0178] In one implementation manner, the range determination module 302 is specifically used to: determine the picture range that needs to be analyzed on the online course screen according to the video characteristics of the online course video based on the corresponding relationship between a plurality of preset video characteristics and the picture range.

[0179] The video characteristics may be online course type or online course screen layout, etc. The online course type refers to the category to which the online course video belongs. The category may include several course subjects or several course themes, such as junior high school Chinese, senior high school English, test question explanation topics, review topics, etc. The online course screen layout refers to the arrangement and planning of each element displayed on the online course screen, such as the position and size of each element on the online course screen.

[0180] For courses of the same type (such as the same course subject or the same course theme, etc.), during the online course video playback, the things that need to be identified and analyzed will basically appear in a fixed size and fixed position on the online course screen. Therefore, when the video feature is the online course type, the range determination module 302 can be specifically used to pre-set a number of course types corresponding to the screen range that needs to be analyzed on the online course screen, and determine the screen range that needs to be analyzed on the online course screen according to the course type to which the online course video belongs.

[0181] For the online course screen layout, the screen layout may include the overall screen layout of the online course screen, and may also include the internal screen layout of each element constituting the overall screen layout. Therefore, when the video feature is the online course screen layout, the range determination module 302 may be specifically used to pre-set a number of screen layouts corresponding to the screen ranges that need to be analyzed on the online course screen, and determine the screen range that needs to be analyzed on the online course screen according to the screen layout of the online course video.

[0182] In another embodiment, the screen range that needs to be analyzed on the online course screen can be determined based on the analysis range circled by the user, which can not only enhance the interactivity between the online course video and the user during the online course video playback, but also make the subsequent analysis results more in line with the user's extended learning needs. At this time, the range determination module 302 can be specifically used to obtain the analysis range circled by the user on the online course screen, and determine the screen range that needs to be analyzed on the online course screen according to the preset range determination rules and the analysis range.

[0183] More specifically, the range determination module 302 can be used to obtain the analysis range circled by the user on the online course screen, and based on the preset shape, the smallest screen on the online course screen with the preset shape and including the analysis range is used as the screen range to be analyzed. The user can be the student watching the online course video, or the parent or teacher who guides the student to watch the online course video.

[0184] After the recommendation display module 304 obtains the content category of the image, a two-step recommendation strategy can be adopted: first determine the recommendation category, then determine the recommended content, thereby improving the accuracy of the recommendation. Accordingly, at this time, the recommendation display module 304 can specifically include a content category determination unit 341, a recommendation category determination unit 342, a recommended content determination unit 343 and a recommended content display unit 344.

[0185] A content category determination unit 341, configured to determine a content category of a picture according to the content of the picture;

[0186] A recommendation category determination unit 342, configured to determine a recommendation category according to the content category, wherein the recommendation category includes one or more elements;

[0187] A recommended content determination unit 343, configured to select one or more elements from the recommended category as recommended content;

[0188] The recommended content display unit 344 is used to display the recommended content when the online course video is played.

[0189] The content category determination unit 341 can be specifically used to: input the image into a multi-label classification model, determine the possibility of whether the image contains each content category, output the probability of whether the image contains each content category, and when the probability is greater than a preset probability threshold, use the content category corresponding to the probability as the content category of the image, or input the image into a multi-classification model, determine the possibility of the image belonging to each content category, output the probability of the image belonging to each content category, and use the content category corresponding to the maximum probability as the content category of the image.

[0190] The recommended content determination unit 343 can be specifically used to: extract feature information of elements and pictures respectively, where the feature information includes one or more of the following: color, shape; and select one or more elements from the recommendation category as recommended content according to the matching degree of the feature information of the elements and pictures.

[0191] In order to capture things that can be analyzed and identified from the online course video in real time and continuously during the playback of the online course video, so as to make real-time and continuous content recommendations, the picture acquisition module 301 is specifically used to capture the currently playing online course pictures multiple times at a fixed rate when the online course video is playing, each time capturing an online course picture displayed at the current moment.

[0192] The image generation module 303 captures a portion of the online course screen for each collected online course screen according to the screen range determined by the range determination module 302, and generates a picture from the captured portion of the screen. For one online course screen, one screen range corresponds to a portion of the screen captured to generate one picture.

[0193] If the difference between the two pictures generated by the same screen range of the two online course pictures collected twice is very small, it is not necessary to repeat the analysis, and directly reuse the recommended content determined by the analysis of the previous picture, thereby reducing the analysis time, speeding up the speed and experiencing fluency. Figure 4 As shown, the device provided by this embodiment may further include a difference detection module 305 .

[0194] The difference detection module 305 is used to perform a difference detection between the currently generated picture and the picture corresponding to the same picture range generated last time to obtain a difference degree if the currently generated picture is not the first picture generated corresponding to the picture range; and determine, according to the difference degree, whether the content category determination unit 41 determines the content category of the currently generated picture according to the content of the currently generated picture;

[0195] The content category determination unit 341 is specifically used to determine the content category of the currently generated picture based on the content of the currently generated picture if the currently generated picture is the first picture generated corresponding to the screen range, or the difference detection module 305 determines the content category of the currently generated picture based on the content of the currently generated picture.

[0196] The recommended content display unit 344 is specifically used for determining the content category of the generated picture based on the content of the online course screen collected this time, then determining the recommended content corresponding to the online course screen according to the content category, and displaying the recommended content when the online course video is played; if the content category of the generated picture is not determined based on the content of the generated picture, then displaying the recommended content corresponding to the online course screen collected before this time when the online course video is played.

[0197] The implementation principle and technical effects of the device for content interaction based on online course videos provided in this embodiment are similar to those of the method embodiment, and will not be repeated here.

[0198] This embodiment also provides a computer device, which can be structured as follows: Figure 5As shown, it includes a processor 410 and a memory. The memory includes a non-volatile storage medium 420 and an internal memory 430. The non-volatile storage medium 420 stores an operating system 421 and a computer program 422. The internal memory 430 provides an environment for the operation of the operating system 421 and the computer program 422 in the non-volatile storage medium 420.

[0199] The processor 410 is used to provide computing and control capabilities. When the computer program 422 is executed by the processor 410, the method for content interaction based on online course videos described in Example 1 or Example 2 is implemented. The processor 410 can communicate with external devices through the I / O interface 440.

[0200] The external device may include a display 510, which may be a liquid crystal display or an electronic ink display, etc. The processor 410 is connected to the display 510 via an I / O interface 440. When implementing the content interaction method, the processor 410 may send the determined recommended content to the display 510 for display via the I / O interface 440.

[0201] The external device may further include a playback device 520. When implementing the content interaction method, the playback device 520 is used to play the online course video. When the online course video is played, the collected online course screen is sent to the processor 410 through the I / O interface 440.

[0202] The display 510 can be used to simultaneously display the online course video played by the playback device 520 and the recommended content determined when the computer device implements the content interaction method, or it can be used only to display the recommended content determined when the computer device implements the content interaction method, and the online course video is displayed by another display configured by the playback device 520.

[0203] The external device may further include an input device 530, which may be a touch layer covered on the display 510, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc. When implementing the content interaction method, the user's input may be received to implement interaction with the user.

[0204] The implementation principle and technical effects of the computer device provided in this embodiment are similar to those of the method embodiment, and will not be described in detail here.

[0205] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 or Embodiment 2 are implemented.

[0206] The implementation principle and technical effect of the computer-readable storage medium provided in this embodiment are similar to those of the method embodiment and will not be repeated here.

[0207] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for content interaction based on online course videos, characterized in that: include: When the online course video is playing, the online course screen is collected, and the online course screen is a static screen displayed at a moment when the online course video is playing; When the online course video is played, the online course screen is collected multiple times at a fixed rate, and each time, one of the online course screens displayed at the current moment is collected; Based on a preset range determination rule, determining a screen range that needs to be analyzed on the online course screen; Obtain the analysis scope circled by the user on the online course screen; Based on a preset range determination rule, according to the analysis range, determining a screen range to be analyzed on the online course screen; According to the screen range, a partial area of ​​the online course screen is intercepted, and a picture is generated from the intercepted partial area; Determine the content category of the picture according to the content of the picture; For one of the online course screens, one of the screen ranges corresponds to capturing a partial area of ​​the online course screen to generate a picture; If the currently generated picture is the first picture generated corresponding to the picture range, determining the content category of the currently generated picture according to the content of the currently generated picture; If the currently generated picture is not the first picture generated corresponding to the picture range, a difference detection is performed between the currently generated picture and the picture corresponding to the same picture range generated last time to obtain the degree of difference; Determining, according to the magnitude of the difference, whether to determine the content category of the currently generated picture according to the content of the currently generated picture; The degree of difference is calculated as follows: extract information from the currently generated image and the last generated image to form information matrices Img1 and Img2, and calculate the degree of difference ε using the following formula: ε = Mean (Img1 - Img2); In the above formula, Mean() is a function that calculates the average value of each element in the matrix; Alternatively, the degree of difference is judged according to the size, position and number of regions having differences between the currently generated image and the last generated image; Determine recommended content according to the content category, and display the recommended content when the online course video is played.

2. A method for content interaction based on online course videos according to claim 1, characterized in that: Based on the preset range determination rule, according to the analysis range, determining the screen range to be analyzed in the online course screen includes: Based on the preset shape, the smallest screen having the preset shape and including the analysis range is used as the screen range to be analyzed in the online course screen.

3. The method for content interaction based on online course videos according to claim 1, characterized in that: Determining recommended content according to the content category, and displaying the recommended content when the online course video is played, including: For the online course screen collected this time, if the content category of the generated picture is determined according to the content of the generated picture, the recommended content corresponding to the online course screen is determined according to the content category, and the recommended content is displayed when the online course video is played; If the content category of the generated image is not determined based on the content of the generated image, the recommended content corresponding to the online course screen collected before this time will be displayed when the online course video is played.

4. The method for content interaction based on online course videos according to claim 1, characterized in that: Determining the content category of the picture according to the content of the picture includes: Input the image into a multi-label classification model, determine the possibility of the image containing each content category according to the content of the image, and output the probability of whether the image contains each content category; When the probability is greater than a preset probability threshold, taking the content category corresponding to the probability as the content category of the picture; Alternatively, the image is input into a multi-classification model, the possibility of the image belonging to each content category is determined according to the content of the image, the probability of the image belonging to each content category is output, and the content category corresponding to the maximum probability is used as the content category of the image.

5. The method for content interaction based on online course videos according to claim 1, characterized in that: Determining recommended content according to the content category includes: Determine a recommendation category according to the content category, wherein the recommendation category includes one or more elements; One or more elements are selected from the recommendation category as recommended content.

6. A method for content interaction based on online course videos according to claim 5, characterized in that: Selecting one or more elements from the recommendation category as recommended content includes: Extracting feature information of the element and the picture respectively, wherein the feature information includes one or more of the following: color and shape; One or more of the elements are selected from the recommendation category as recommended content according to a matching degree between the feature information of the element and the picture.

7. A device for content interaction based on online course videos, characterized in that: include: The picture acquisition module is used to collect the online class picture when the online class video is played, and the online class picture is a static picture displayed at a certain moment when the online class video is played; A range determination module, used to determine the screen range that needs to be analyzed in the online course screen based on a preset range determination rule; A picture generation module, used to capture a partial area of ​​the online course screen according to the screen range, and generate a picture of the captured partial area; The recommendation display module, for one of the online course pictures, a picture range corresponds to intercepting a part of the online course picture to generate a picture; If the currently generated picture is the first picture generated corresponding to the picture range, determining the content category of the currently generated picture according to the content of the currently generated picture; If the currently generated picture is not the first picture generated corresponding to the screen range, a difference detection is performed between the currently generated picture and the picture corresponding to the same screen range generated last time to obtain a degree of difference. The recommendation display module is used to determine, according to the size of the degree of difference, whether to determine the content category of the currently generated picture based on the content of the currently generated picture, determine the content category of the picture based on the content of the picture, determine the recommended content based on the content category, and display the recommended content when the online course video is played.

8. A computer device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method for content interaction based on online course video as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for content interaction based on online course videos as described in any one of claims 1 to 6 is implemented.

Images