Video analysis method and device

Abstract

The invention relates to a video analysis method and device, and the method comprises the steps: selecting teaching original data according to the time correlation of the teaching original data in response to the obtained teaching original data, obtaining target teaching data, processing the target teaching data through employing a teaching ST algorithm, and obtaining a first video analysis result, receiving a second video analysis result transmitted by the server, and fitting the second video analysis result and the first video analysis result through a fitting algorithm to obtain a final video analysis result; by adopting the method, the first video analysis result can be quickly obtained without waiting for video analysis completion under the condition that the server is in an abnormal state, and the final video analysis result can be output after the server is recovered, so that the real-time performance and reliability of video analysis are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video analysis method and device. Background Art

[0002] The real-time performance of existing video analysis mainly depends on algorithms and server computing power, and it is often necessary to wait for the video analysis to be completed before obtaining the first video analysis result.

[0003] However, the real-time performance of video analysis is easily affected by the video size and other factors. In the event of an abnormality in the server, the real-time performance and reliability of video analysis will be greatly affected. Summary of the invention

[0004] Based on this, it is necessary to provide a video analysis method and device that can improve the real-time and reliability of video analysis in response to the above technical problems.

[0005] In a first aspect, the present application provides a video analysis method, the method comprising:

[0006] In response to obtaining the original teaching data, the original teaching data is selected according to the time correlation of the original teaching data to obtain target teaching data;

[0007] The teaching ST algorithm is used to process the target teaching data to obtain the first video analysis result;

[0008] Receiving a second video analysis result transmitted by the server; the second video analysis result is obtained by processing the video stream data by the server using the teaching ST algorithm;

[0009] The second video analysis result and the first video analysis result are fitted by a fitting algorithm to obtain a final video analysis result.

[0010] In one of the embodiments, the original teaching data includes a pilot code time sequence;

[0011] According to the time correlation of the original teaching data, the original teaching data is selected to obtain the target teaching data including:

[0012] According to the video time, the director code of the corresponding time period in the director code time sequence is selected as the target teaching data.

[0013] In one embodiment, the original teaching data is selected according to the time correlation of the original teaching data to obtain the target teaching data, including:

[0014] Based on the video time or the video start time corresponding to the original teaching data, the original teaching data of the target time period is filtered out;

[0015] The original teaching data of the target time period is used as the target teaching data.

[0016] In one embodiment, the method further comprises:

[0017] In response to the target teaching data being a plurality of video segments, obtaining time periods corresponding to the respective video segments;

[0018] The repeated video segments that overlap each time period within the video time period are deleted from the target teaching data.

[0019] In one embodiment, the method further comprises:

[0020] Take the union of each time period to get the video time period.

[0021] In one embodiment, the second video analysis result and the first video analysis result are fitted by a fitting algorithm to obtain a final video analysis result, including:

[0022] Selecting a corresponding fitting function according to the characteristics of the second video analysis result and the first video analysis result;

[0023] The fitting function is used to merge the second video analysis result and the first video analysis result to obtain and output the teaching model data; the teaching model data is used to instruct the terminal to draw and generate a teaching ST diagram.

[0024] In a second aspect, the present application further provides a video analysis device, the device comprising:

[0025] A data acquisition module, for selecting the original teaching data according to the time correlation of the original teaching data in response to acquiring the original teaching data, to obtain target teaching data;

[0026] A result generation module, used for processing target teaching data using a teaching ST algorithm to obtain a first video analysis result;

[0027] A receiving module, used for receiving a second video analysis result transmitted by a server; the second video analysis result is obtained by processing the video stream data by the server using a teaching ST algorithm;

[0028] The fitting module is used to fit the second video analysis result and the first video analysis result through a fitting algorithm to obtain a final video analysis result.

[0029] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0030] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0031] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0032] The above-mentioned video analysis method and device, in response to obtaining the original teaching data, selects the original teaching data according to the time correlation of the original teaching data to obtain the target teaching data, and uses the teaching ST algorithm to process the target teaching data to obtain the first video analysis result, and receives the second video analysis result transmitted by the server, and fits the second video analysis result and the first video analysis result through a fitting algorithm to obtain the final video analysis result; when the server is in an abnormal state, the application can quickly obtain the first video analysis result without waiting for the server to complete the video analysis, and can output the final video analysis result after the server is restored, thereby improving the real-time and reliability of the video analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A diagram showing an application environment of a video analysis method in an embodiment;

[0034] Figure 2 A schematic diagram of a video analysis method according to an embodiment of the present invention;

[0035] Figure 3 is a schematic flow chart of a video analysis method in another embodiment;

[0036] Figure 4 is a logic flow chart of a video analysis method in one embodiment;

[0037] Figure 5 is a logic flow chart corresponding to a logic algorithm in one embodiment;

[0038] Figure 6 is a structural block diagram of a video analysis device in one embodiment;

[0039] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0042] It is understood that terms such as "first" and "second" in this application are only used to distinguish similar objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. The "connection" appearing in the embodiments of this application refers to various connection methods such as direct connection or indirect connection to achieve communication between devices, and the embodiments of this application do not make any limitation on this.

[0043] It will be understood that “at least one” means one or more, and “plurality” means two or more.

[0044] When used herein, the singular forms "a", "an" and " / the" may also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" and the like specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0045] The theory of ST model analysis comes from a paper analyzing the behavior of classroom teachers and students in the field of education. This theory can be applied to data analysis. Based on the real-time analysis of the ST model of video, the real-time performance of video analysis currently depends mainly on algorithms and server computing power. It is often necessary to wait until the server analyzes the video before the result can be obtained. The real-time performance will be affected by various factors such as the size of the video, and there are unreliability such as server downtime and high server load. These have a great impact on the real-time performance and reliability of video analysis, and thus affect the user experience. Among them, S in the ST model can refer to Student, and T can refer to Teacher.

[0046] The present application can be used in conventional scenarios such as ST model analysis of videos. By adopting the teaching ST algorithm, the first video analysis result can be obtained without waiting for the server to complete the video analysis. After receiving the second video analysis result output by the server after completing the video analysis, the first video analysis result and the second video analysis result are fitted to obtain the final video analysis result, thereby improving the real-time and reliability of the analysis.

[0047] The video analysis method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 and the server 104 can both communicate with the cloud service application 106, the camera can collect the behavior of the user (students and teachers) to obtain the original teaching data and upload it to the server 104, the server 104 transmits the original teaching data to the cloud service application 106, the cloud service application 106 selects the original teaching data according to the time correlation of the original teaching data, obtains the target teaching data, and uses the teaching ST algorithm to process the target teaching data to obtain the first video analysis result, the server 104 receives the video stream data output by the camera, and uses the teaching ST algorithm to process the video stream data, the server 104 outputs the processed second video analysis result to the cloud service application 106, the cloud service application 106 fits the second video analysis result and the first video analysis result through the fitting algorithm to obtain the final video analysis result, and the terminal 102 can be used to view the first video analysis result and the final video analysis result. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 may be implemented as an independent server or a server cluster composed of multiple servers. Furthermore, the server 104 may be an AI (Artificial Intelligence) server. The cloud service application 106 may be configured on the cloud to implement an application for the terminal 102 to interact with the cloud.

[0048] In one embodiment, Figure 2 As shown, a video analysis method is provided, which is applied to Figure 1 Take the cloud service application in as an example to illustrate the method, which includes:

[0049] S202, in response to obtaining the original teaching data, selecting the original teaching data according to the time correlation of the original teaching data to obtain target teaching data.

[0050] Among them, the original teaching data may include student data and teacher data; among them, teacher data may represent teacher behavior in classroom teaching, and student data may represent student behavior in classroom teaching, that is, all behaviors in classroom teaching except teacher behavior.

[0051] Specifically, in response to obtaining the original teaching data, the cloud service application can select the original teaching data according to a logical algorithm linked to time and the time correlation of the original teaching data, obtain target teaching data, and improve the real-time performance of video analysis.

[0052] It should be noted that the logic algorithm linked to time may include multiple logics, which are not described in detail in the embodiments of the present application; the logic algorithm can be used to select data from the original teaching data according to the corresponding time logic.

[0053] In actual applications, the cloud service application can link the obtained ST rough data (student data and teacher data) with time through a time-linked logical algorithm to select the target teaching data.

[0054] In one embodiment, the target teaching data is the data of the corresponding time period in the original teaching data;

[0055] The method also includes:

[0056] Receive the original teaching data transmitted by the server.

[0057] Specifically, the target teaching data may be data selected from the original teaching data through a logical algorithm; the server may obtain the original teaching data collected by the camera, and transmit the original teaching data to the cloud service application.

[0058] It should be noted that the data corresponding to some time periods in the original teaching data may not be highly correlated with the teaching quality. Logical algorithms can be used to select target teaching data.

[0059] In the embodiment of the present application, the reliability of video analysis is improved by receiving the original teaching data transmitted by the server and selecting data of the corresponding time period from the original teaching data as target data.

[0060] In one embodiment, the original teaching data is selected according to the time correlation of the original teaching data to obtain the target teaching data, including:

[0061] Based on the video time or video start time corresponding to the original teaching data, filter out the original teaching data of the target time period;

[0062] The original teaching data of the target time period is used as the target teaching data.

[0063] Among them, the target time period can be set according to actual conditions and is not limited in the embodiments of the present application.

[0064] Specifically, the cloud service application can obtain the video time corresponding to the teaching original data, filter out the teaching original data of the target time period from the video time according to the logical algorithm, and use the teaching original data of the target time period as the target teaching data; the cloud service application can also determine the video start time corresponding to the teaching original data, and use the teaching original data with a preset time interval from the video start time as the target teaching data according to the logical algorithm.

[0065] It should be noted that the preset interval duration can be set according to actual conditions and is not limited in the embodiments of the present application.

[0066] In the embodiment of the present application, the target teaching data is determined based on the video time or the video start time corresponding to the original teaching data, thereby improving the reliability of the video analysis.

[0067] In one of the embodiments, the original teaching data includes a pilot code time sequence;

[0068] According to the time correlation of the original teaching data, the original teaching data is selected to obtain the target teaching data including:

[0069] According to the video time, the director code of the corresponding time period in the director code time sequence is selected as the target teaching data.

[0070] Among them, the director code can refer to a string composed of letters, numbers and / or characters. Different director codes can represent different behavioral meanings, and the behavioral meanings include the specific content of the behavior and the goal of issuing the behavior. The design basis of the director code can be combined with specific application scenarios. For example, in the teaching recording scenario, involving teachers and students, the director code can be used to represent teacher T behavior (explanation, demonstration, media display, questioning, evaluation and feedback) and student S behavior (speaking, thinking and taking notes).

[0071] Among them, the director time series can be expressed as sampling the original teaching data according to a certain time interval, and according to the behavior type of the sample point, the corresponding symbol S (Student) or T (Teacher) is recorded to form the ST time series data; for example, a 45-minute class can be sampled at intervals of 30 seconds, and a 15-minute class can be sampled every 10 seconds. If the moment is a teacher's behavior, T is recorded in the table, otherwise S is recorded, and these symbols constitute an ST data series (ST time series data).

[0072] Specifically, the cloud service application can select the director code of the corresponding time period in the director code time sequence according to the video time based on the logical algorithm, and use the director code as the target teaching data.

[0073] In an embodiment of the present application, the director code of the corresponding time period in the director code time sequence is selected according to the video time as the target teaching data, thereby improving the reliability of the video analysis.

[0074] In one embodiment, the method further comprises:

[0075] In response to the target teaching data being a plurality of video segments, obtaining time periods corresponding to the respective video segments;

[0076] The repeated video segments that overlap each time period within the video time period are deleted from the target teaching data.

[0077] Specifically, the target teaching video can be multiple video segments. The cloud service application can obtain the time periods corresponding to each video segment, and delete the repeated video segments that overlap each time period within the video time period from the target teaching data, so as to reduce the amount of subsequent data processing.

[0078] In the embodiment of the present application, by deleting repeated video segments that overlap each time period within the target teaching data, the amount of subsequent processing data is reduced while improving the reliability of video analysis.

[0079] In one embodiment, the method further comprises:

[0080] Take the union of each time period to get the video time period.

[0081] Specifically, the cloud service application may determine the video time period by taking the union of each time period.

[0082] Exemplarily, each time period may include a first time period and a second time period. The cloud service application may take the union of the first time period and the second time period to obtain a third time period of repeated video segments. The video time period may be obtained after deleting the third time period.

[0083] In the embodiment of the present application, the video time period is determined by taking the union of each time period, thereby improving the reliability of video analysis.

[0084] S204, using the teaching ST algorithm to process the target teaching data to obtain a first video analysis result.

[0085] Among them, the teaching ST algorithm can refer to the ST analysis method, namely the Student-Teacher analysis method, which can be used for quantitative analysis of the teaching process and represent the teaching results in a graphical manner, so that teachers can use a visual method to discuss the teaching process; further, the first video analysis result obtained using the teaching ST algorithm can also be called an ST model.

[0086] Specifically, the cloud service application can use the teaching ST algorithm to process the target teaching data and quickly obtain the first video analysis result. The first video analysis result can be represented by an ST graph. It should be noted that the time consumed in the process of obtaining the first video analysis result can reach the millisecond level, which improves the real-time performance of the video analysis.

[0087] For example, the ST diagram can take the origin as the starting time of teaching, with S as the vertical axis and T as the horizontal axis, which respectively represent the time of S behavior and T behavior. The measured S and T data (target teaching data) are sequentially represented on the S axis and T axis, and the length of each axis can be set to 45 minutes (one class period), and a complete ST diagram is obtained. Through the ST diagram, it can be seen how the teacher's behavior and the student's behavior change over time during the whole teaching process, and the teaching situation of teachers and students can be seen, which is convenient for analyzing the advantages and disadvantages in their teaching activities, so that further corrections can be made to the teaching design and teaching behavior.

[0088] S206, receiving a second video analysis result transmitted by the server; the second video analysis result is the video stream data processed by the server using the teaching ST algorithm.

[0089] Specifically, the camera can capture video streams of users (students and teachers) and transmit the captured video streams to the server in real time. The server can use the teaching ST algorithm to analyze and process the video streams, and output the second video analysis results to the cloud service application. It should be noted that the second video analysis results can be more detailed and complete than the first video analysis results.

[0090] S208, fitting the second video analysis result and the first video analysis result by using a fitting algorithm to obtain a final video analysis result.

[0091] Among them, the fitting algorithm can be set according to actual conditions and is not limited in the embodiments of the present application.

[0092] Specifically, the cloud service application may fit the second video analysis result and the first video analysis result according to a fitting algorithm to obtain a final video analysis result.

[0093] For example, the problem of data fluctuations and large differences causing ugly icons (final video analysis results) can be solved by function fitting (fitting algorithm). This process improves the real-time performance of video analysis and reduces unnecessary waiting for the AI ​​server to finish analyzing the video.

[0094] In actual applications, if there is an abnormal situation in the server, the cloud service application can output the first video analysis result without waiting for the server to output the second video analysis result. The approximate result will not be unable to be seen in time due to abnormal factors such as server crashes. After the server is restored, analysis and processing can be carried out in time. After obtaining the second video analysis result, the cloud server can fit the second video analysis result and the first video analysis result through a fitting algorithm to obtain the final video analysis result.

[0095] Optionally, the fitting algorithm may be performed by using a mathematical formula or an open source library to process the second video analysis result and the first video analysis result.

[0096] In one embodiment, if Figure 3 As shown, the second video analysis result and the first video analysis result are fitted by a fitting algorithm to obtain a final video analysis result, including:

[0097] S302: Select a corresponding fitting function according to the features of the second video analysis result and the first video analysis result.

[0098] The fitting function can be set according to actual conditions and is not limited in this application.

[0099] Specifically, the cloud service application may first obtain features of the second video analysis result and the first video analysis result, and select a corresponding fitting function according to the features.

[0100] Exemplarily, the features of the second video analysis result and the first video analysis result may refer to target type, action type, action number, and corresponding time.

[0101] S304, using a fitting function to merge the second video analysis result and the first video analysis result, to obtain and output teaching model data; the teaching model data is used to instruct the terminal to draw and generate a teaching ST diagram.

[0102] Specifically, the cloud service application can use a fitting function to merge the analysis results of the second video and the analysis results of the first video to obtain ST sequence data (teaching model data). The terminal can draw a teaching ST diagram, where the origin of the teaching ST diagram can be the start time of the teaching; the measured S and T data are sequentially represented on the S / T axis until the end of the teaching.

[0103] In an embodiment of the present application, a selected fitting function is used to merge the second video analysis result and the first video analysis result to obtain and output teaching model data to instruct the terminal to draw and generate a teaching ST diagram, thereby improving the real-time and reliability of video analysis.

[0104] In the above-mentioned video analysis method, in response to obtaining the original teaching data, the original teaching data is selected according to the time correlation of the original teaching data to obtain the target teaching data, and the teaching ST algorithm is used to process the target teaching data to obtain the first video analysis result, and the second video analysis result transmitted by the receiving server is received, and the second video analysis result is fitted with the first video analysis result through a fitting algorithm, so as to obtain the final video analysis result. When the server is in an abnormal state, there is no need to wait for the server to complete the video analysis, and the first video analysis result can be quickly obtained. After the server is restored, the final video analysis result can be output, thereby improving the real-time and reliability of the video analysis.

[0105] In one embodiment, the method further comprises:

[0106] The target teaching data is stored in the database.

[0107] Specifically, the cloud service application can store the target teaching data in a database to realize the backup of the target teaching data and improve the reliability of video analysis.

[0108] In the embodiment of the present application, the reliability of video analysis is improved by backing up the target teaching data.

[0109] In one embodiment, the database is a ClickHouse database.

[0110] Specifically, since the target teaching data may be large and time-related (related to time), it is necessary to select a corresponding storage database suitable for big data to improve the real-time performance of video analysis.

[0111] Optionally, the ClickHouse database is used as an example in the embodiments of the present application for illustration, which is not intended to limit the present application.

[0112] To facilitate understanding by those skilled in the art, the video analysis method is described below with reference to two specific examples. Figure 4 and Figure 5 As shown, users can refer to teachers and students; PC side can refer to terminals.

[0113] In one example, Figure 4As shown, in the traditional solution, the video-based ST real-time analysis is to upload the video and then analyze it through the AI ​​server to obtain ST data and analyze the results (the second video analysis result). Here, the camera and the AI ​​server divide the collection work. The camera can collect the user's behavior data (teaching original data), and the AI ​​server analyzes other data other than behavior (video stream data), where the video stream data is the video stream collection of the user through the camera, and outputs it to the AI ​​server in real time; the AI ​​server reports the acquired ST collection data (teaching original data) and starts to analyze the ST data (video stream data) of the video. After the data captured by the camera (teaching original data) is reported to the cloud (cloud service application) through the server, the cloud service application can first quickly simulate the approximate ST model diagram (the first video analysis result) through the time-linked logical algorithm simulation, and store the behavior data (target teaching data) in the ClickHouse database. This real-time performance is very high, generally reaching the millisecond level. This real-time performance depends on ClickHouse storage and time-linked logical algorithms.

[0114] After the AI ​​server completes the analysis and reports the video analysis results (the second video analysis results), the cloud service application will supplement the detailed data (the second video analysis results) into the ST model diagram (the first video analysis results) through various function fitting methods. The ST diagram is eventually perfected to obtain the final result (the final video analysis result). In this way, the real-time performance of video analysis is improved. There is no need to wait for the AI ​​server to analyze the video and obtain the analysis results. There is also no possibility of not being able to see the approximate results in time due to abnormal factors such as AI server crashes. After recovery, the ST model can be repaired in time, and the reliability of video analysis is guaranteed.

[0115] In another example, Figure 5 As shown, Figure 5 This is a logical flow chart of the time-linked algorithm based on video analysis, which is explained by taking the original teaching data as the director code. It should be noted that each director code can have a corresponding director screen, where the director screen is the screen taken by the camera and recorded by the recorder.

[0116] The recorder can upload the director code and recorded video to the device service. The device service can output the director code to the ClickHouse database. The cloud service application can select the director code of the corresponding time period in the director code time series of a certain device (select the director code of the corresponding time period according to the video time), and perform ST model analysis on it (use the teaching ST algorithm to process the target teaching data), and output charts (the analysis results of the first video).

[0117] The device service can output the director code to the MySQL database. The cloud service application can also obtain the recorded video uploaded to the platform and only process the director video. If the classroom video has multiple segments, take the start time and end time as the union of all segment times, select the corresponding time period director code based on the start time and end time, and perform ST model analysis on it (use the teaching ST algorithm to process the target teaching data), and output charts (first video analysis results).

[0118] It should be noted that the director video can refer to the video generated when a recorder is used for directing. The director code is the peripheral of the recorder. For example, the original data processed by the camera to recognize the characteristics of students or teachers in the directing scene is called the director code, that is, the original teaching data.

[0119] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a video analysis device for implementing the video analysis method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more video analysis device embodiments provided below can refer to the limitations on the video analysis method above, and will not be repeated here.

[0121] In one embodiment, Figure 6 As shown, a video analysis device 600 is provided, the device comprising:

[0122] The data acquisition module 601 is used for selecting the original teaching data according to the time correlation of the original teaching data in response to obtaining the original teaching data to obtain the target teaching data;

[0123] A result generating module 602 is used to process the target teaching data using the teaching ST algorithm to obtain a first video analysis result;

[0124] The receiving module 603 is used to receive the second video analysis result transmitted by the server; the second video analysis result is the video stream data processed by the server using the teaching ST algorithm;

[0125] The fitting module 604 is used to fit the second video analysis result and the first video analysis result by using a fitting algorithm to obtain a final video analysis result.

[0126] In one embodiment, the data acquisition module 601 is further used to filter out the original teaching data of the target time period based on the video time corresponding to the original teaching data;

[0127] The original teaching data of the target time period is used as the target teaching data.

[0128] In one of the embodiments, the original teaching data includes a director code time sequence; the data acquisition module 601 is further used to select a director code of a corresponding time period in the director code time sequence according to the video time as target teaching data.

[0129] In one embodiment, the data acquisition module 601 is further used to filter out the original teaching data of the target time period based on the video time or the video start time corresponding to the original teaching data;

[0130] The original teaching data of the target time period is used as the target teaching data.

[0131] In one embodiment, the video analysis device 600 further includes a deletion module, which is used to obtain time periods corresponding to the respective video segments in response to the target teaching data being a plurality of video segments;

[0132] The repeated video segments that overlap each time period within the video time period are deleted from the target teaching data.

[0133] In one of the embodiments, the deletion module is further used to obtain the union of each time period to obtain the video time period.

[0134] In one embodiment, the fitting module 604 is further used to select a corresponding fitting function according to the characteristics of the second video analysis result and the first video analysis result;

[0135] The fitting function is used to merge the second video analysis result and the first video analysis result to obtain and output the teaching model data; the teaching model data is used to instruct the terminal to draw and generate a teaching ST diagram.

[0136] Each module in the above video analysis device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0137] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target teaching data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video analysis method is implemented.

[0138] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned video analysis method when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned video analysis method is implemented.

[0141] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above-mentioned video analysis method when executed by a processor.

[0142] It should be noted that the data involved in this application (including but not limited to original teaching data, video streaming data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video analysis method, characterized in that: The method comprises: In response to acquiring the original teaching data, the original teaching data is selected according to the time correlation of the original teaching data to obtain target teaching data; Using the teaching ST algorithm to process the target teaching data to obtain a first video analysis result; Receiving a second video analysis result transmitted by the server; the second video analysis result is obtained by processing the video stream data by the server using the teaching ST algorithm; The second video analysis result and the first video analysis result are fitted by a fitting algorithm to obtain a final video analysis result.

2. The method according to claim 1, characterized in that The original teaching data includes a time sequence of a broadcasting code; The selecting the original teaching data according to the time correlation of the original teaching data to obtain the target teaching data includes: According to the video time, the director code of the corresponding time period in the director code time sequence is selected as the target teaching data.

3. The method according to claim 1, characterized in that The selecting the original teaching data according to the time correlation of the original teaching data to obtain the target teaching data includes: Based on the video time or the video start time corresponding to the original teaching data, the original teaching data of the target time period is filtered out; The original teaching data of the target time period is used as the target teaching data.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: In response to the target teaching data being a plurality of video segments, obtaining time periods corresponding to the respective video segments; The repeated video segments in which the time periods overlap within the video time period are deleted from the target teaching data.

5. The method according to claim 4, characterized in that The method further comprises: The union of the time periods is taken to obtain the video time period.

6. The method according to claim 1, characterized in that The step of fitting the second video analysis result and the first video analysis result by a fitting algorithm to obtain a final video analysis result includes: Selecting a corresponding fitting function according to the characteristics of the second video analysis result and the first video analysis result; The fitting function is used to merge the second video analysis result and the first video analysis result to obtain and output teaching model data; the teaching model data is used to instruct the terminal to draw and generate a teaching ST diagram.

7. A video analysis device, characterized in that: The device comprises: A data acquisition module, configured to, in response to acquiring the original teaching data, select the original teaching data according to the time correlation of the original teaching data to obtain target teaching data; A result generating module, used for processing the target teaching data by using the teaching ST algorithm to obtain a first video analysis result; A receiving module, used for receiving a second video analysis result transmitted by a server; the second video analysis result is obtained by processing the video stream data by the server using the teaching ST algorithm; The fitting module is used to fit the second video analysis result and the first video analysis result by using a fitting algorithm to obtain a final video analysis result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images