AI intelligent resource processing method and system

Abstract

The invention discloses an AI intelligent resource processing method and system. Comprising the following steps: firstly, acquiring multi-source information including a teacher camera, a student camera, a computer picture, a teacher teaching sound and a blackboard writing picture, and collecting and storing the multi-source information as historical collection information; thirdly, carrying out sound and picture fusion on the teaching sound information of the teacher and the camera picture and computer picture information of the teacher and the students to generate a fusion media stream; streaming media coding is carried out on the fusion media stream to obtain coding information; then, according to a network environment and platform requirements, the coding information is pushed to a school direct recording and broadcasting course platform; and finally, performing progress evaluation according to the historical acquisition information and the coding information to obtain a resource processing progress evaluation value, optimizing a resource processing scheme according to the resource processing progress evaluation value, and realizing classroom content live broadcast and course recording and broadcasting resource accumulation. According to the invention, the problem of asynchronous traditional sound and picture is solved, flexible adjustment can be realized according to teaching situations, resource utilization is optimized, stable and smooth transmission is guaranteed, cost is reduced, and efficiency is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of resource processing, and in particular to an AI intelligent resource processing method and system. BACKGROUND

[0002] In the current technical environment, various traditional streaming media encoders have great limitations in function implementation. They only mechanically and singly perform encoding operations on input video signals and audio signals, and the encoding process often lacks flexibility and intelligence. For example, when facing complex video scenes or audio environments, targeted encoding optimization cannot be performed according to specific content characteristics.

[0003] With the continuous iteration and upgrading of AI technology, intelligent resource processing has entered the information field of education, conferences and the like, and the traditional streaming media encoder cannot flexibly open or close the push according to the timetable of the school classroom. If the push is maintained all the time, the network bandwidth resources of the school will be occupied.

[0004] Moreover, only single-channel video signals and audio signals are supported, and the input of multi-channel signals is not supported. When there are multiple picture sources and sound sources in the classroom, they cannot be simultaneously accessed and processed, and are subject to the hardware architecture. The intelligent resource processing method and system do not have secondary processing, effect optimization algorithms and capabilities for picture and sound signals, and the overall output effect is general.

[0005] Therefore, an AI intelligent resource processing method and system are needed to solve the above problems. SUMMARY

[0006] The application aims to provide an AI intelligent resource processing method, which comprises the following steps:

[0007] Multiple source information in a classroom is obtained and stored as historical collection information, and the multiple source information comprises teacher camera picture information, student camera picture information, computer picture information, teacher teaching sound information and blackboard picture information;

[0008] The teacher teaching sound information is subjected to audio and picture fusion processing with the teacher camera picture information, the student camera picture information and the computer picture information to generate a fusion media stream;

[0009] The fusion media stream is subjected to streaming media encoding to obtain encoding information;

[0010] According to network environment information and platform requirement information, the encoding information is pushed to a school direct recording and broadcasting course platform through a network;

[0011] According to the historical collection information and the encoding information, progress evaluation is performed to obtain a resource processing progress evaluation value;

[0012] Optimize the resource processing scheme based on the resource processing progress evaluation value to achieve live streaming of classroom content and accumulation of recorded course resources.

[0013] Furthermore, the step of comprehensively collecting the multi-source information includes:

[0014] The system collects camera footage from teachers and students using multiple cameras installed in the classroom.

[0015] Using computer screen capture technology to acquire information from computer screen images;

[0016] Collect teacher's audio information based on audio acquisition equipment;

[0017] The data acquisition module of the intelligent writing board acquires information about the writing screen;

[0018] The collected information is tagged with timestamps and stored as historical information.

[0019] Furthermore, the audio-visual fusion processing steps include:

[0020] Audio preprocessing is performed on the teacher's lecture audio information, including noise reduction, gain adjustment, and audio format conversion;

[0021] The system performs image preprocessing on the teacher's camera footage, the student's camera footage, and the computer screen footage. The processing steps include image cropping, resolution adjustment, and color correction.

[0022] According to the preset synchronization rules, the pre-processed audio information of the teacher's lecture is synchronized with the information of each screen to ensure audio-visual synchronization.

[0023] The synchronized audio and video information of the teacher's lecture are combined according to the preset layout rules to generate a merged media stream. The layout rules can be adjusted according to the user's needs to change the display position and size of the screen.

[0024] Furthermore, the step of pushing the encoded fused media stream to the school's live-streaming course platform via the network includes:

[0025] Analyze network environment information, including network bandwidth, latency, and packet loss rate metrics;

[0026] Based on the platform's requirements, determine the necessary transmission protocols, such as RTMP and HLS.

[0027] Based on the network metrics and platform transmission protocols obtained from the analysis, the encoded converged media stream is transmitted over the network.

[0028] The transmission status of the push process is monitored in real time, and error correction is performed if any abnormalities occur (such as network interruption or transmission failure).

[0029] Furthermore, the step of performing progress assessment based on historically collected information and encoded information to obtain a resource processing progress assessment value includes:

[0030] Extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources;

[0031] Data on push speed, push data volume, and push success rate are obtained based on the encoded information.

[0032] Based on the collected data and encoding information, a progress evaluation model is established, which considers the progress of multiple stages, including data collection, audio-visual fusion, encoding, and push.

[0033] The resource processing progress assessment value is calculated using the aforementioned progress assessment model. This assessment value is used to reflect the speed and stability of the entire resource processing process.

[0034] Furthermore, the step of optimizing the resource processing scheme based on the resource processing progress assessment value includes:

[0035] Set standard values ​​for resource processing progress;

[0036] Compare the resource processing progress assessment value with the resource processing progress standard value;

[0037] If the resource processing progress assessment value is lower than the standard value, analyze the performance indicators of each stage to optimize the resource processing plan. Optimization measures include adjusting the acquisition equipment parameters, improving the audio-visual fusion algorithm, changing the encoding strategy, and optimizing the network transmission settings.

[0038] If the resource processing progress assessment value reaches or exceeds the standard value, the plan will be fine-tuned based on historical processing data to further improve processing efficiency and quality and achieve continuous optimization.

[0039] This invention also discloses an AI intelligent resource processing system, comprising:

[0040] The first acquisition module is used to acquire multi-source information in the classroom, including teacher camera footage, student camera footage, computer screen footage, teacher's audio information, and blackboard information.

[0041] The acquisition module is used to collect the multi-source information from all directions and store it as historical acquisition information;

[0042] The fusion module is used to perform audio-visual fusion processing on the teacher's audio information, the teacher's camera footage, the student's camera footage, and the computer screen information to generate a fused media stream;

[0043] The encoding module is used to perform streaming media encoding on the converged media stream and obtain encoding information;

[0044] The push module is used to push encoded information to the school's live-streaming course platform via the network, based on network environment information and platform requirements.

[0045] The second acquisition module is used to perform progress assessment based on historical collection information and encoding information, and obtain resource processing progress assessment value;

[0046] The optimization module is used to optimize the resource processing scheme based on the resource processing progress evaluation value, so as to realize the live streaming of classroom content and the accumulation of recorded course resources.

[0047] Furthermore, the second acquisition module includes:

[0048] The extraction unit is used to extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources.

[0049] The acquisition unit is used to obtain push speed, push data volume, and push success rate data based on the encoded information.

[0050] The evaluation unit is used to integrate the above-mentioned collected data and encoding information to establish a progress evaluation model. This model considers the progress of multiple stages, including data collection, audio-visual fusion, encoding, and push.

[0051] The calculation unit is used to calculate the resource processing progress assessment value using the progress assessment model. The assessment value is used to reflect the speed and stability of the entire resource processing process.

[0052] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0053] This application also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0054] The beneficial effects of this application are as follows:

[0055] Firstly, the information collection of this invention is comprehensive and intelligent, covering multiple information sources in the classroom, collecting and storing information from all angles, providing rich data and long-term support for teaching analysis. The audio-visual integration is accurate and adaptable to different scenarios, solving the problem of audio-visual asynchrony in traditional methods. It can be flexibly adjusted according to teaching or meeting situations, improving the experience and resource quality. Meanwhile, the streaming media encoding is flexible and efficient, adapting to network and platform requirements, optimizing resource utilization, ensuring stable and smooth transmission, reducing costs and improving efficiency.

[0056] Secondly, the progress evaluation and optimization mechanism of this invention is comprehensive, monitors the process status in real time, and continuously optimizes the plan based on the evaluation value to ensure the efficient and stable operation of the system, adapt to changes in teaching and technology, provide strong technical support for education and conference informatization, and help the development of smart education, making it more innovative and practical. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a method flow proposed in an embodiment of this application.

[0058] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] This application provides an AI-powered intelligent resource processing method, including the following steps:

[0061] S1, acquire multi-source information in the classroom and store it as historical data. The multi-source information includes teacher camera footage, student camera footage, computer screen footage, teacher's audio, and blackboard information.

[0062] S2, collect the multi-source information from all directions and store it as historical collection information;

[0063] S3 integrates the teacher's audio information with the teacher's camera footage, the student's camera footage, and the computer screen footage to generate a fused media stream;

[0064] S4, perform streaming media encoding on the fused media stream to obtain encoding information;

[0065] S5, based on network environment information and platform requirements, pushes the encoded information to the school's live-streaming course platform via the network;

[0066] S6. Based on historical data and coding information, perform progress assessment and obtain resource processing progress assessment value;

[0067] S7. Optimize the resource processing scheme based on the resource processing progress evaluation value to realize the live streaming of classroom content and the accumulation of recorded course resources.

[0068] As described in steps S1-S7 above, the multi-source information of this invention has a wide coverage and can acquire information from various sources in the classroom, including teacher's camera footage, student's camera footage, computer screen information, teacher's audio information, and blackboard writing information. It comprehensively covers all aspects of classroom teaching. Compared with the traditional approach that only focuses on a single or a few information sources, it can record the classroom situation more completely and provide a rich data foundation for subsequent teaching analysis and resource reuse.

[0069] Collecting information from multiple sources in a comprehensive manner and storing it as historical data not only facilitates subsequent review and retrieval, but also provides long-term data support for teaching quality assessment and student learning behavior analysis. This helps educators gain a deeper understanding of the teaching process and students' learning situation, thereby enabling targeted teaching improvements and personalized instruction.

[0070] This method integrates the teacher's audio with various visual elements to create a merged media stream. This solves the problems of audio-visual asynchrony or low correlation in traditional methods, providing viewers with a more immersive experience, as if they were actually there. This helps improve the learning effectiveness and engagement of remote learners, while also ensuring the high-quality presentation of teaching resources. This fusion approach can flexibly adjust the combination of audio and visuals according to different teaching scenarios and needs. For example, when explaining key knowledge points, the teacher's visuals and audio can be highlighted; during group discussions, student visuals and interactive audio can be emphasized, better meeting diverse teaching situations and enhancing the relevance and practicality of teaching resources.

[0071] This invention can also perform streaming media encoding on converged media streams based on network environment information and platform requirements. It can automatically select or adjust the most suitable encoding parameters and formats according to actual network conditions (such as bandwidth and latency) and the technical requirements of the target platform. This ensures that the encoded information maintains high quality during network transmission while adapting to different network conditions, avoiding video stuttering and blurry images caused by network issues, and improving the stability and smoothness of resource transmission. Through reasonable encoding methods, the media stream is effectively compressed and optimized, reducing data volume and lowering the demand for network bandwidth and storage resources without affecting information integrity and quality. This is beneficial for improving the overall performance and resource utilization of the system, especially in large-scale educational resource distribution and storage scenarios, significantly reducing costs and improving efficiency.

[0072] Based on historical data collection and encoding information, progress assessment is performed to obtain resource processing progress evaluation values. This allows for real-time monitoring of the entire resource processing flow, enabling timely detection of potential problems such as data acquisition equipment failure, low encoding efficiency, and network transmission latency. This provides crucial data and reference for system operation and management, facilitating proactive intervention and optimization. Optimizing resource processing schemes based on progress evaluation values ​​enables the system's self-adjustment and continuous improvement. Analysis of these evaluation values ​​allows for targeted adjustments to data acquisition parameters, fusion algorithms, encoding strategies, and push mechanisms, continuously optimizing the entire resource processing flow. This ensures the system maintains consistently high efficiency and stability, adapting to evolving teaching needs and technological environments, and providing reliable technical support and assurance for live classroom content creation and the accumulation of recorded course resources.

[0073] Specifically, the steps for comprehensively collecting the multi-source information include:

[0074] The system collects camera footage from teachers and students using multiple cameras installed in the classroom.

[0075] Using computer screen capture technology to acquire information from computer screen images;

[0076] Collect teacher's audio information based on audio acquisition equipment;

[0077] The data acquisition module of the intelligent writing board acquires information about the writing screen;

[0078] The collected information is tagged with timestamps and stored as historical information.

[0079] Through the aforementioned data collection steps, specialized collection methods are employed for different types of information sources. For example, multiple camera devices are used to capture images of teachers and students, computer screen capture technology is used to obtain computer images, audio capture devices are used to collect teachers' lectures, and a smart writing board data acquisition module is used to capture blackboard writing. This ensures the accuracy and professionalism of various information collection methods, comprehensively capturing every detail of the classroom. The collected information is timestamped and stored as historical data, facilitating subsequent management, retrieval, and analysis. The timestamping provides a clear chronological order to the information, allowing for quick location of relevant information at specific points in time when reviewing classroom content, conducting teaching assessments, or studying student learning processes. This improves the usability and efficiency of information utilization, providing strong support for refined management and in-depth research in education and teaching. The entire process is complete and clear, allowing users to understand the operation methods and information flow paths of the entire collection process, which is conducive to the practical application and promotion of this technology.

[0080] Specifically, the audio-visual fusion processing steps include:

[0081] Audio preprocessing is performed on the teacher's lecture audio information, including noise reduction, gain adjustment, and audio format conversion;

[0082] The system performs image preprocessing on the teacher's camera footage, the student's camera footage, and the computer screen footage. The processing steps include image cropping, resolution adjustment, and color correction.

[0083] According to the preset synchronization rules, the pre-processed audio information of the teacher's lecture is synchronized with the information of each screen to ensure audio-visual synchronization.

[0084] The synchronized audio and video information of the teacher's lecture are combined according to the preset layout rules to generate a merged media stream. The layout rules can be adjusted according to the user's needs to change the display position and size of the screen.

[0085] It should be noted that preprocessing the teacher's audio information, including noise reduction, gain adjustment, and audio format conversion, effectively removes environmental noise interference, making the sound clearer and purer. Gain adjustment ensures appropriate volume, while audio format conversion facilitates subsequent integration and transmission, laying the foundation for high-quality audio-visual integration. Preprocessing of the teacher's and students' webcam images, as well as the computer screen, involves cropping, resolution adjustment, and color correction. Cropping removes unnecessary parts of the image, highlighting key content; resolution adjustment adapts to different display devices and transmission requirements, ensuring image clarity; and color correction makes the colors more realistic and natural, improving the overall visual effect and enjoyment of the image.

[0086] This invention performs audio-visual synchronization processing according to preset synchronization rules, ensuring accurate matching between the teacher's voice and various screen information. It avoids the problem of poor viewing experience caused by audio-visual asynchrony in traditional methods, enabling viewers to obtain a coherent and smooth audio-visual experience and enhancing the professionalism and usability of teaching resources.

[0087] The synchronized audio and video information is combined according to the preset layout rules to generate a fused media stream. The layout rules can be adjusted to adjust the display position and size of the screen according to user needs. This flexibility can meet the needs of different teaching scenarios and user preferences. For example, relevant screens can be highlighted when explaining key content, and multiple student screens can be flexibly displayed during group discussions. This improves the relevance and adaptability of teaching resources, making the audio and video fusion process more scientific and refined. It can generate high-quality fused media streams that meet user needs, providing strong support for applications in the field of education and teaching.

[0088] Specifically, the step of pushing the encoded fused media stream to the school's live-streaming course platform via the network includes:

[0089] Analyze network environment information, including network bandwidth, latency, and packet loss rate metrics;

[0090] Based on the platform's requirements, determine the necessary transmission protocols, such as RTMP and HLS.

[0091] Based on the network metrics and platform transmission protocols obtained from the analysis, the encoded converged media stream is transmitted over the network.

[0092] The transmission status of the push process is monitored in real time, and error correction is performed if any abnormalities occur (such as network interruption or transmission failure).

[0093] Specifically, the step of performing progress assessment and obtaining resource processing progress assessment values ​​based on historically collected information and encoded information includes:

[0094] Extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources;

[0095] Data on push speed, push data volume, and push success rate are obtained based on the encoded information.

[0096] Based on the collected data and encoding information, a progress evaluation model is established, which considers the progress of multiple stages, including data collection, audio-visual fusion, encoding, and push.

[0097] Let the data acquisition progress be P. c The audio-visual integration progress is P. h The coding progress is P. e The push progress is P. p The resource processing progress assessment value is P, and the data collection weight is W. c The weight for audio-visual integration is W. h The encoding weight is W. e The push weight is W p And W c +W h +W e +W p =1, then: the formula for calculating the resource processing progress assessment value is:

[0098]

[0099] The resource processing progress assessment value is calculated using the aforementioned progress assessment model. This assessment value is used to reflect the speed and stability of the entire resource processing process.

[0100] The numerator W in the above formula c P c +W h P h +W e Pe +W p P p It is the sum of the products of the progress of each stage and its corresponding weight, taking into account the progress and importance of each stage. The denominator is... The denominator plays a certain role in adjusting the calculation results. Its main function is to normalize the numerator's calculation results, ensuring that the evaluation value P is within a reasonable range and avoiding excessive fluctuations in the evaluation value due to differences in weight values. This makes the calculation results more scientific and reasonable, and can more accurately reflect the actual situation of the resource processing process.

[0101] W c +W h +W e +W p =1, so the importance of each stage can be flexibly adjusted according to the actual situation. In the case of an unstable network environment, the weight of the push stage can be appropriately increased, because the stability of the push is crucial to whether the final resources can be delivered effectively. In scenarios with extremely high requirements for picture quality, the weight of audio-visual integration can be increased accordingly to highlight its role in ensuring the viewing experience.

[0102] It is important to note that data collection should be extracted from historical data sources, including the collection time, data volume, and collection frequency from different information sources (teacher's camera footage, student's camera footage, computer screen footage, teacher's audio, and blackboard footage). This data should be used to reflect the progress of data collection. For example, the progress can be determined by comparing the ratio of the amount of data collected to the expected amount of data to be collected, and by assessing the completion rate based on the collection frequency and time.

[0103] During the audio-visual fusion process, the completion status of each processing step (audio preprocessing, video preprocessing, synchronization processing, and combining to generate a fused media stream) is recorded and evaluated. For example, the proportion of preprocessed audio and video data is recorded, and the progress of audio-visual fusion is measured based on the completion status of synchronization processing and combining to generate a fused media stream. The progress of audio-visual fusion can also be determined by statistically analyzing the proportion of processed audio and video data to the total data volume.

[0104] During the streaming media encoding process of converged media streams, the amount of data already encoded and the encoding time information are recorded. The encoding progress is reflected by the ratio of the amount of data already encoded to the total amount of data to be encoded. The degree of completion of encoding is evaluated based on the encoding speed (the amount of data encoded per unit time) and the encoding time, thereby determining the encoding progress.

[0105] The push speed, push data volume, and push success rate are obtained based on the encoding information. The push progress is determined by the ratio of the pushed data volume to the total data volume to be pushed. Alternatively, the push completion status can be comprehensively evaluated by combining the push speed and push time, while also considering the impact of the push success rate on the push progress.

[0106] By employing this processing method, when pushing the encoded and fused media stream to the school's live-streaming course platform, network environment information, including bandwidth, latency, and packet loss rate, is analyzed first. This allows for the selection of appropriate transmission strategies based on actual network conditions, ensuring stable transmission of the media stream under different network conditions. This improves the system's adaptability and compatibility with network environments. Furthermore, by determining the required transmission protocols, such as RTMP and HLS, based on platform requirements, seamless integration with different platforms is ensured, avoiding push failures or playback issues caused by protocol incompatibility and enhancing the accuracy and reliability of the push.

[0107] This invention enables real-time monitoring of the transmission status during the push process. In the event of network interruptions or transmission failures, it can promptly correct errors, ensuring the continuity and stability of the push process and reducing the interruption or loss of teaching resources due to push failures, thus providing users with a smoother user experience. It extracts data on collection time, data volume, and collection frequency from different information sources from historical data collection, and simultaneously obtains push speed, push data volume, and push success rate data based on encoding information, covering multiple stages of resource processing and providing comprehensive and detailed data support for progress evaluation.

[0108] The aforementioned calculation model for resource processing progress assessment integrates data from multiple sources, establishing a progress assessment model that considers the progress of multiple stages, including data acquisition, audio-visual fusion, encoding, and delivery. This model provides a comprehensive and systematic evaluation of the entire resource processing process, avoiding the limitations of single-indicator evaluations and enabling the assessment results to more accurately reflect the overall system performance and operational status. The resource processing progress assessment value calculated using this model can intuitively reflect the speed and stability of the entire resource processing process, providing an important basis for system optimization and management. Through this assessment value, managers can quickly locate system problems and then take targeted measures for improvement and optimization, thereby enhancing system operating efficiency and resource processing quality.

[0109] These advantages make it highly practical and reliable in media streaming and resource processing progress assessment, better meeting the educational sector's demand for high-quality, stable live-streamed course resources.

[0110] Specifically, the step of optimizing the resource processing plan based on the resource processing progress assessment value includes:

[0111] Set standard values ​​for resource processing progress;

[0112] Compare the resource processing progress assessment value with the resource processing progress standard value;

[0113] If the resource processing progress assessment value is lower than the standard value, analyze the performance indicators of each stage to optimize the resource processing plan. Optimization measures include adjusting the acquisition equipment parameters, improving the audio-visual fusion algorithm, changing the encoding strategy, and optimizing the network transmission settings.

[0114] If the resource processing progress assessment value reaches or exceeds the standard value, the plan will be fine-tuned based on historical processing data to further improve processing efficiency and quality and achieve continuous optimization.

[0115] By setting standard values ​​for resource processing progress, a clear and quantifiable target is provided for the entire resource processing process. This gives the system's operation and optimization a clear reference standard, allowing operators and managers to intuitively understand the gap between the system's current operating status and the expected goals. Comparing the resource processing progress assessment value with the standard value allows for a quick and accurate determination of whether the system's overall performance and efficiency meet the requirements. This comparative analysis method is simple and intuitive, enabling timely identification of potential system problems and providing a basis and direction for subsequent optimization measures.

[0116] When the evaluated value falls below the standard value, performance indicators at each stage are analyzed to identify problematic areas, and targeted optimization measures are taken, such as adjusting acquisition device parameters, improving audio-visual fusion algorithms, changing encoding strategies, and optimizing network transmission settings. This phased and targeted optimization approach can accurately solve problems, effectively improve the efficiency and quality of resource processing, and avoid the waste of resources and insignificant results caused by blind optimization. Even if the evaluated value reaches or exceeds the standard value, the solution is fine-tuned based on historical processing data to further improve processing efficiency and quality, achieving continuous optimization. This continuous improvement mechanism enables the invention to constantly adapt to new needs and environmental changes, maintain good operating status and performance improvement, and enhance adaptability and competitiveness.

[0117] This invention also discloses an AI intelligent resource processing system, comprising:

[0118] The first acquisition module is used to acquire multi-source information in the classroom, including teacher camera footage, student camera footage, computer screen footage, teacher's audio information, and blackboard information.

[0119] The acquisition module is used to collect the multi-source information from all directions and store it as historical acquisition information;

[0120] The fusion module is used to perform audio-visual fusion processing on the teacher's audio information, the teacher's camera footage, the student's camera footage, and the computer screen information to generate a fused media stream;

[0121] The encoding module is used to perform streaming media encoding on the converged media stream and obtain encoding information;

[0122] The push module is used to push encoded information to the school's live-streaming course platform via the network, based on network environment information and platform requirements.

[0123] The second acquisition module is used to perform progress assessment based on historical collection information and encoding information, and obtain resource processing progress assessment value;

[0124] The optimization module is used to optimize the resource processing scheme based on the resource processing progress evaluation value, so as to realize the live streaming of classroom content and the accumulation of recorded course resources.

[0125] Furthermore, the second acquisition module includes:

[0126] The extraction unit is used to extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources.

[0127] The acquisition unit is used to obtain push speed, push data volume, and push success rate data based on the encoded information.

[0128] The evaluation unit is used to integrate the above-mentioned collected data and encoding information to establish a progress evaluation model. This model considers the progress of multiple stages, including data collection, audio-visual fusion, encoding, and push.

[0129] The calculation unit is used to calculate the resource processing progress assessment value using the progress assessment model. The assessment value is used to reflect the speed and stability of the entire resource processing process.

[0130] It is worth mentioning that, regarding the acquisition of blackboard image information through the smart writing board data acquisition module, compared to ordinary encoders, the smart writing board in this invention can use a set of infrared photosensitive modules installed around the perimeter of an ordinary blackboard to form the smart writing board data acquisition module mentioned above. It determines the movement trajectory coordinate data of the chalk within the board surface through a dense grating. The fusion module has multiple interfaces, thereby acquiring the blackboard trajectory coordinate data of multiple smart writing boards. Then, after acquiring the trajectory coordinate data, it refits it into blackboard image information and encodes it into a network stream for local HDMI image output. This is something that existing encoders cannot do.

[0131] It's important to note that regarding the refitting of acquired trajectory coordinate data into blackboard image information, the infrared sensor module can form a dense grating on the smart writing board (composed of a regular blackboard with an added infrared sensor module). When chalk writes on the blackboard, the infrared sensor module uses the grating to determine the chalk's movement trajectory within the board surface, thereby acquiring the chalk's trajectory coordinate data. This data accurately records the changes in each position of the chalk on the blackboard. Next, the fusion module has multiple interfaces, enabling it to acquire blackboard trajectory coordinate data from multiple smart writing boards. After acquiring this data, the system processes these discrete coordinate data based on a specific algorithm. For example, a Bézier curve fitting algorithm (a current technology) can be used, which generates a smooth curve by defining control points. When fitting the blackboard trajectory, the collected discrete coordinate points can be used as control points or used to calculate control points. Based on the Bézier curve formula, the algorithm uses these control points to calculate other points on the curve, thus connecting the discrete chalk trajectory coordinate data into a smooth curve to simulate blackboard strokes. For example, when drawing curved shapes or cursive characters on the blackboard, Bézier curves can make the strokes transition naturally, conform to writing habits, and make the fitted blackboard writing look more realistic and smooth.

[0132] This algorithm connects discrete points based on the temporal order and spatial relationship of coordinates, refitting them into continuous lines. These lines constitute the strokes of the blackboard writing. Many strokes combine to form a complete blackboard image. Finally, the fitted blackboard image is encoded into a network stream and then output locally via HDMI. This allows the blackboard image to be displayed on various projectors and commercial display devices, facilitating student viewing and solving the problem of students in the back rows not being able to see the blackboard clearly.

[0133] The whiteboard information from the HDMI signal can be output to various projectors and commercial display devices. In addition, a large screen or projection device can be set up in the middle of the classroom, which can enlarge the writing on the blackboard so that students in the back rows can see it. In this way, students can view the PPT on one screen and the whiteboard on another screen, thus solving the problem of not being able to see clearly due to the different distances between students' seats.

[0134] It is important to note that, when collecting teacher's audio information using audio acquisition equipment, compared to ordinary encoders that can only input a single audio signal, the audio acquisition device in this invention includes multiple pickups and multiple microphones. Furthermore, the audio acquisition device can simultaneously process the input of these multiple audio signals. Since the use of amplification equipment by teachers during lectures is highly random—they may use a microphone or not—the pickup effect is clearer when a microphone is used. However, if a microphone is not used, there is no signal input at the microphone end, resulting in poor audio quality from pickups alone. This invention, by simultaneously acquiring audio content from both microphones and pickups, can compare and analyze the two audio signals, and then fuse and optimize them to ensure that the audio content is more clearly superimposed onto the screen, accurately recording the audio information of the lecture.

[0135] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0136] This application also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0139] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent results or equivalent process transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An AI-powered intelligent resource processing method, characterized in that, Includes the following steps: Acquire multi-source information from within the classroom and store it as historical data. The multi-source information includes teacher camera footage, student camera footage, computer screen footage, teacher's audio, and blackboard information. The audio information of the teacher's lecture is combined with the video information from the teacher's camera, the video information from the student's camera, and the computer screen to generate a fused media stream. Perform streaming media encoding on the fused media stream to obtain encoding information; Based on network environment information and platform requirements, the encoded information is pushed to the school's live-streaming course platform via the network; Based on historical data and coding information, progress assessment is performed to obtain resource processing progress assessment values; Optimize the resource processing scheme based on the resource processing progress evaluation value to achieve live streaming of classroom content and accumulation of recorded course resources.

2. The AI ​​intelligent resource processing method according to claim 1, characterized in that, The steps for comprehensively collecting the multi-source information include: The system collects camera footage from teachers and students using multiple cameras installed in the classroom. Using computer screen capture technology to acquire information from computer screen images; Collect teacher's audio information based on audio acquisition equipment; The data acquisition module of the intelligent writing board acquires information about the writing screen; The collected information is tagged with timestamps and stored as historical information.

3. The AI ​​intelligent resource processing method according to claim 1, characterized in that, The steps of the audio-visual fusion processing include: Audio preprocessing is performed on the teacher's lecture audio information, including noise reduction, gain adjustment, and audio format conversion; The system performs image preprocessing on the teacher's camera footage, the student's camera footage, and the computer screen footage. The processing steps include image cropping, resolution adjustment, and color correction. According to the preset synchronization rules, the pre-processed audio information of the teacher's lecture is synchronized with the information of each screen to ensure audio-visual synchronization. The synchronized audio and video information of the teacher's lecture are combined according to the preset layout rules to generate a merged media stream.

4. The AI ​​intelligent resource processing method according to claim 1, characterized in that, The steps of pushing the encoded fused media stream to the school's live-streaming course platform via the network include: Analyze network environment information, including network bandwidth, latency, and packet loss rate metrics; Based on the platform's requirements, determine the necessary transmission protocol for the platform; Based on the network metrics and platform transmission protocols obtained from the analysis, the encoded converged media stream is transmitted over the network. The transmission status during the push process is monitored in real time, and error correction is performed if any abnormalities occur.

5. The AI ​​intelligent resource processing method according to claim 4, characterized in that, The step of performing progress assessment and obtaining resource processing progress assessment values ​​based on historically collected information and encoded information includes: Extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources; Data on push speed, push data volume, and push success rate are obtained based on the encoded information. Based on the collected data and coding information, a progress assessment model is established, and the resource processing progress assessment value is calculated and obtained through the progress assessment model. Let the data acquisition progress be P. c The audio-visual integration progress is P. h The coding progress is P. e The push progress is P. p The resource processing progress assessment value is P, and the data collection weight is W. c The weight for audio-visual integration is W. h The encoding weight is W. e The push weight is W p And W c +W h +W e +W p =1, then: the formula for calculating the resource processing progress assessment value is: The resource processing progress assessment value is calculated using the above progress assessment model. This assessment value is used to reflect the speed and stability of the entire resource processing process.

6. The AI ​​intelligent resource processing method according to claim 5, characterized in that, The step of optimizing the resource processing scheme based on the resource processing progress assessment value includes: Set standard values ​​for resource processing progress; Compare the resource processing progress assessment value with the resource processing progress standard value; If the resource processing progress assessment value is lower than the standard value, analyze the performance indicators of each stage to optimize the resource processing plan. Optimization measures include adjusting the acquisition equipment parameters, improving the audio-visual fusion algorithm, changing the encoding strategy, and optimizing the network transmission settings. If the resource processing progress assessment value reaches or exceeds the standard value, the plan will be fine-tuned based on historical processing data to further improve processing efficiency and quality and achieve continuous optimization.

7. An AI-powered intelligent resource processing system, characterized in that, include: The first acquisition module is used to acquire multi-source information in the classroom, including teacher camera footage, student camera footage, computer screen footage, teacher's audio information, and blackboard information. The acquisition module is used to collect the multi-source information from all directions and store it as historical acquisition information; The fusion module is used to perform audio-visual fusion processing on the teacher's audio information, the teacher's camera footage, the student's camera footage, and the computer screen information to generate a fused media stream; The encoding module is used to perform streaming media encoding on the converged media stream and obtain encoding information; The push module is used to push encoded information to the school's live-streaming course platform via the network, based on network environment information and platform requirements. The second acquisition module is used to perform progress assessment based on historical collection information and encoding information, and obtain resource processing progress assessment value; The optimization module is used to optimize the resource processing scheme based on the resource processing progress evaluation value, so as to realize the live streaming of classroom content and the accumulation of recorded course resources.

8. The AI ​​intelligent resource processing system according to claim 7, characterized in that, The second acquisition module includes: The extraction unit is used to extract collected data from historical information, including the collection time, data volume, and collection frequency of different information sources. The acquisition unit is used to obtain push speed, push data volume, and push success rate data based on the encoded information. The evaluation unit is used to integrate the collected data and coded information to establish a progress evaluation model. The calculation unit is used to calculate the resource processing progress assessment value using the progress assessment model. The assessment value is used to reflect the speed and stability of the entire resource processing process.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the control method for the offshore wind power anti-scour device based on digital twin technology as described in claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for the offshore wind power anti-scour device based on digital twin technology as described in claims 1-6.

Images