AI-based integrated recording and broadcasting dotting and playing method

Abstract

The application discloses an integrated recording and broadcasting dotting and playing method based on AI. During a class, a recording and broadcasting system collects audio and video data information of the class through an audio and video collecting device, performs compression coding, and then transmits the audio and video data information to a previously configured ftp server address through a network to complete a video storage operation. During the class, dynamic identification of knowledge points is performed to separately record and store the knowledge points. The dynamic identification relies on a desktop capturing sensor, infrared sensing, camera capturing, and an autonomous triggering system to capture knowledge point data and store the knowledge point data into a knowledge point array. The knowledge point data is uploaded through an automatic ftp resource uploading mode and is synchronously uploaded with the video, and the storage operation of the knowledge points and the video is completed. A cloud platform player renders a video progress dotting according to the video and the knowledge point file. A client logs in the cloud platform of the recording and broadcasting system, can watch online, and can check the knowledge points through the dotting and playing, so that the learning focus can be found through the dotting and playing of the knowledge points.

Description

TECHNICAL FIELD

[0001] The present application relates to audio and video intelligent recording and broadcasting technology, and in particular to an integrated recording and broadcasting dotting and playing method based on AI. BACKGROUND

[0002] The recording and broadcasting system can record and synchronize the video, audio, and image signals of electronic devices (including computers, video booths, etc.) recorded on site, generate standardized streaming media files, and use them for external live broadcast, storage, post-editing, and on-demand viewing. The recording and broadcasting system is organically combined with school teaching, and signals of high-quality courses and important meetings are collected by multiple camera positions and recorded to a computer hard disk. Students or teachers can watch the video files from different camera angles. At the same time, online information sources are sorted and uploaded to achieve the purpose of fully utilizing network learning resources.

[0003] Traditional recording and broadcasting equipment needs to record video, copy resources through a U disk, edit video through a teacher's computer, mark key points in the video progress bar, and then upload the video to a resource platform for release, so that students can watch the video online.

[0004] Each classroom is equipped with a recording and broadcasting system, and a large number of on-demand videos are generated, which requires a large amount of manpower for resource management and integration. Post-editing and release of the video require high video editing skills from school teachers, and the arrangement of knowledge points is more complicated and inconvenient for students to quickly find learning focuses after class. SUMMARY

[0005] To solve the problems in the prior art, the present application provides an integrated recording and broadcasting dotting and playing method based on AI.

[0006] To achieve the purpose of the present application, the technical solution adopted by the present application is as follows:

[0007] An integrated recording and broadcasting dotting and playing method based on AI, comprising the steps of:

[0008] (1) After entering the classroom scene, the recording and broadcasting system obtains the recording and broadcasting storage information platform address returned by the cloud platform, i.e., the ftp server address;

[0009] (2) During the class, the recording and broadcasting system collects audio and video data information of the class content through multiple audio and video collection devices, compresses and encodes the audio and video data information, and then transmits the compressed audio and video data information to the pre-configured ftp server address through the network for storage, completing the video storage operation;

[0010] (3) In the process of class, through the dynamic identification of knowledge points, the knowledge points are recorded and stored individually; the dynamic identification relies on the desktop capture sensor, infrared sensor, camera capture, and autonomous trigger system to capture knowledge point data, and store it as a knowledge point array;

[0011] (4) The knowledge point array corresponding to the video generates knowledge point data api, which is uploaded automatically through the ftp resource, and is synchronized with the video; it is stored in the ftp server address configured in advance, and the knowledge point and video are stored in the database.

[0012] (5) The cloud platform player renders the video progress dot according to the video and knowledge point file; the client logs in the cloud platform of the recording and broadcasting system, and can watch online and view knowledge points through dot playing.

[0013] Further, before step (1), the registration and login of the recording and broadcasting system are also included, the teacher enters the cloud platform address through the network end, inputs the school account and password after logging into the system; after successful login, the corresponding grade and class are selected, and the corresponding timetable information and the current course taught by the teacher are obtained.

[0014] Further, in step (3), when the knowledge point area on the smart blackboard is clicked during the class, the desktop capture sensor on the smart blackboard obtains a trigger signal, triggers the system to capture the knowledge point, and the system converts the knowledge point voice data spoken by the teacher into text through language recognition technology, and saves the knowledge point voice and text.

[0015] Further, in step (3), when the blackboard is written during the class, the blackboard infrared sensing area of the intelligent blackboard obtains a trigger signal, triggers the system to capture the blackboard content, and the system converts the blackboard knowledge point image data into text through image recognition technology, and saves the knowledge point image and text.

[0016] Further, in step (3), when the student stands up to speak during the class, the recording camera dynamic capture technology recognizes the student standing up trigger system signal, triggers the system to capture the knowledge point, and the system converts the student speaking knowledge point voice data into text through language recognition technology, and saves the knowledge point voice and text.

[0017] Further, in step (3), the knowledge points are stored in the local hard disk of the recording and broadcasting system in the form of key-value pair data structure, and are saved in correspondence with the video time node of the recording and broadcasting.

[0018] Further, in step (3), a video contains multiple knowledge points, and multiple knowledge points form an array; a video corresponds to a knowledge point array, and the array stores the knowledge points in time stamp.

[0019] Further, in step (4), the knowledge points and the video file are uploaded in real time, at leisure time or at regular time through ftp.

[0020] Further, in step (5), the AI-generated knowledge points during the recording and broadcasting process are presented in the form of pictures and texts on the player, so that the user can know all the knowledge points in advance during the video playing process and quickly locate the knowledge point playing position.

[0021] The present application has the advantages that, compared with the prior art, the present application provides live broadcast, recording and broadcasting and on-demand video integrated services for the client through audio and video information collection, compression, network transmission, storage and management; at the same time, through automatic acquisition, compression and packaging upload of AI knowledge points, students can learn after school and quickly find learning focus through on-demand knowledge points; considering the diversity of the client, the system supports television, computer and mobile phone three-screen integration for all-around user experience. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is the AI-based integrated recording and broadcasting and on-demand playing method flowchart of the present application;

[0023] Figure 2 is a recording and broadcasting registration and login schematic diagram;

[0024] Figure 3 is a smart blackboard knowledge point triggering area schematic diagram;

[0025] Figure 4 is a smart blackboard board book infrared sensing area schematic diagram;

[0026] Figure 5 is a player knowledge point positioning schematic diagram. DETAILED DESCRIPTION

[0027] The technical solutions of the present application will be further described below in combination with the drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0028] As shown in Figure 1 , the AI-based integrated recording and broadcasting and on-demand playing method of the present application comprises the following specific steps:

[0029] As shown in Figure 2 , first, the teacher enters the cloud platform address through the network end, inputs the school account and password after logging into the system, selects the corresponding grade and class after successful login, thereby obtaining the corresponding timetable information and the current course taught by the teacher. The current recording and broadcasting bound class is set according to the classroom to record the course, and the corresponding course is recorded according to the course setting. Figure 3 , Figure 4The video, the dot file generation are synchronously carried out in an inductive mode, and are automatically uploaded to a cloud platform for publishing according to a configured grade and class, and a player of the cloud platform carries out video progress dot rendering according to the video and the dot file, so as to enable students to carry out online learning.

[0030] The teacher clicks to enter the teaching scene, and the system obtains the recording and broadcasting saving information platform address returned by the cloud platform, that is, the ftp server address.

[0031] When the teacher teaches, the system collects audio and video data information of the teaching content through a plurality of video collection devices and audio collection devices, carries out compression coding of the audio and video data information through an embedded network video processor, then transmits the compressed audio and video data information to a streaming media server, and stores the compressed audio and video data information to the pre-configured ftp server address, to complete the video storage operation; meanwhile, the data information is saved in a local hard disk.

[0032] The video signal collection device is composed of three types of cameras, namely, a teacher camera, a student camera and a blackboard camera. Usually, the teacher camera and the student camera adopt a fast spherical camera with a rotating holder, and are respectively aimed at and track the shooting target. The blackboard camera should be able to clearly shoot all the text and graphics written on the blackboard, and can shoot both the local and the whole blackboard.

[0033] In the process of teaching by the teacher, the knowledge points can be recorded and stored individually through dynamic recognition. The dynamic recognition includes relying on a desktop capture sensor, infrared sensing technology and camera capture, so as to autonomously trigger active data capture and storage.

[0034] As shown in Figure 3 , when the teacher clicks the knowledge point area on the smart blackboard in the teaching process, the trigger signal is obtained through the desktop capture sensor on the smart blackboard, so as to trigger the system to capture the knowledge point voice data. The system converts the knowledge point voice data spoken by the teacher at this time into text through language recognition technology, so as to save the voice and text of the knowledge point. When the clicking is stopped, the capture of the voice knowledge point is stopped.

[0035] As shown in Figure 4 , an infrared sensing area is arranged on the smart blackboard for blackboard writing. When the teacher writes on the blackboard by infrared, the signal of writing on the blackboard by the teacher on both sides is obtained through infrared sensing technology, so as to trigger the system to capture the blackboard content, capture the blackboard image information and store it in the knowledge point. The system converts the blackboard knowledge point image data into text through image recognition technology, so as to save the image and text of the knowledge point. When the blackboard writing is stopped, the capture of the blackboard knowledge point is stopped.

[0036] When a student stands up to speak during the recording and broadcasting process, the student's standing and speaking are recognized through dynamic capture technology of the recording and broadcasting camera, a system signal is triggered, the student's standing dynamic image is captured and stored in a specific knowledge point, and at the same time, the system converts the student's speech data on the knowledge point at this time into text through language recognition technology and stores it in the specific knowledge point.

[0037] The knowledge points are stored in the local hard disk of the recording and broadcasting system host in the form of key-value pair data structure, and are saved in correspondence with the video time nodes recorded by the recording and broadcasting. For example, {“20221122”:[{“202211221330”, “This is a learning knowledge point”}]}.

[0038] A video contains multiple knowledge points, and multiple knowledge points form an array; therefore, a video corresponds to a knowledge point array, and the array stores the knowledge points in correspondence with timestamps, which facilitates the video playback platform to play by timestamp.

[0039] Then, the knowledge point array corresponding to the video generates a knowledge point data API, and the knowledge point data is uploaded through an automatic ftp resource uploading method, and is uploaded synchronously with the video; the knowledge point data is stored in the ftp server address configured in advance, and the knowledge point and video storage operation is completed.

[0040] The knowledge points and video files are uploaded through real-time, idle-time and timing uploading methods, and the corresponding knowledge points and video files are stored in the ftp server address configured in advance.

[0041] The files are uploaded to the corresponding class course file directory through the grade and class configured by the recording and broadcasting, the administrator logs in the cloud platform to perform video review and AI dotting data review, and after the review is completed, the publishing operation is performed. The cloud platform player renders the video progress dotting according to the video and dotting file. Students select courses for learning according to the grade and class on the resource platform, and the corresponding knowledge points are displayed in the progress bar.

[0042] The client logs in the recording and broadcasting system cloud platform through the network using the IE browser, and can watch live pictures or on-demand videos in real time without installing any plug-ins, and can view knowledge points through dotting playback.

[0043] Through improvement of the player, users can dot the video at any time through the buttons on the page during video playback, the system receives the user dotting instruction, switches to the dotting time, captures the knowledge point at this time according to the dotting time recorded in the database, and the screen switches to the knowledge point.

[0044] For example, Figure 5As shown, the knowledge points generated by the AI during the recording and broadcasting process are presented in the form of pictures and texts on the player, and the user can know all the knowledge points in advance during the video playing process, quickly locate the knowledge point playing position, play the video, and learn and download the corresponding knowledge points.

[0045] Considering the diversity of the client, the system supports television, computer and mobile phone three screens in one; the user can view the video data of the server in various ways, and achieve cross-terminal and cross-platform user experience.

[0046] The beneficial effects of the present application are that, compared with the prior art, the present application realizes live broadcast, recording and broadcasting, and on-demand video integration services for the client through audio and video information collection, compression, network transmission, storage and management; at the same time, through automatic acquisition, compression and packaging upload of AI knowledge points, students can learn after school and find learning focus faster through on-demand knowledge points; considering the diversity of the client, the system supports television, computer and mobile phone three screens in one for all-round user experience.

[0047] The applicant of the present application has made a detailed description and explanation of the embodiment examples of the present application in combination with the drawings of the specification, but those skilled in the art should understand that the above embodiment examples are only preferred embodiments of the present application, and the detailed description is only to help the reader better understand the spirit of the present application, and is not a limitation on the protection scope of the present application, on the contrary, any improvement or modification based on the spirit of the present application should fall within the protection scope of the present application.

Claims

1. An AI-based integrated recording and broadcasting method, characterized in that: Including steps: (1) After clicking to enter the class scene, the recording and broadcasting system obtains the recording and broadcasting information platform address returned by the cloud platform, that is, the FTP server address; (2) During class, the recording and broadcasting system collects audio and video data of the class content through multiple audio and video acquisition devices, compresses and encodes the audio and video data information, and then transmits the compressed audio and video data information through the network to a pre-configured FTP server address to complete the video storage operation; (3) During class, knowledge points are recorded and stored individually through dynamic recognition of knowledge points; dynamic recognition relies on desktop capture sensors, infrared sensors, camera capture, and autonomous triggering systems to capture knowledge point data and store them as knowledge point arrays; When a user clicks on a knowledge point area on the smart blackboard during class, the desktop capture sensor on the smart blackboard receives a trigger signal, triggering the system to capture the knowledge point. The system uses language recognition technology to convert the teacher's voice data of the knowledge point into text and saves the knowledge point voice and text. When writing on the blackboard during class, the infrared sensing area of ​​the smart blackboard receives a trigger signal, triggering the system to capture the content of the blackboard. The system uses image recognition technology to convert the image data of the knowledge points on the blackboard into text, and saves the images and text of the knowledge points. When a student stands up to speak during class, the recording camera's dynamic capture technology recognizes the student standing up and triggers the system to capture the knowledge points. At the same time, the system uses language recognition technology to convert the student's speech data into text and save the knowledge points voice and text. (4) The knowledge point array corresponding to the video generates the knowledge point data API, and uploads the knowledge point data through the FTP resource automatic upload method, and uploads it synchronously with the video; it is stored in the pre-configured FTP server address, completing the storage operation of the knowledge points and videos; (5) The cloud platform player performs video progress marking and rendering based on the video and knowledge point files; the client logs in to the recording and broadcasting system cloud platform to watch online and view knowledge points through marking and playback.

2. The AI-based integrated recording and broadcasting method according to claim 1 is characterized in that: Before step (1), the system also includes registration and login of the recording and broadcasting system. The cloud platform address is entered through the network terminal, and the school account and password are entered after logging into the system. After successful login, the corresponding grade and class are selected to obtain the corresponding timetable information and the courses taught by the current teacher.

3. The AI-based integrated recording and broadcasting method according to claim 1 is characterized in that: In step (3), the knowledge points are stored in the local hard disk of the recording and broadcasting system in the form of a key-value pair data structure, and are saved in correspondence with the time nodes of the recorded video.

4. The AI-based integrated recording and broadcasting method according to claim 1 is characterized in that: In step (3), a video contains multiple knowledge points, and the multiple knowledge points form an array; one video corresponds to one knowledge point array, and the array stores the knowledge points in correspondence with timestamps.

5. The AI-based integrated recording and broadcasting method according to claim 1, characterized in that: In step (4), knowledge points and video files are uploaded via FTP in real-time, idle time, and scheduled uploading modes.

6. The AI-based integrated recording and broadcasting method according to claim 1 is characterized in that: In step (5), the player will present the knowledge points generated by AI during the recording process in the form of pictures + text. The user can predict all the knowledge points in advance during the video playback and quickly locate the playback position of the knowledge point.

Images