A popular science video extraction system based on edge computing and artificial intelligence analysis
Through edge computing and artificial intelligence analysis technology, key information of popular science videos is extracted and compressed and stored, which solves the problem that existing systems cannot effectively extract popular science video information, and achieves rapid search of video content and reduces the transmission volume.
Patent Information
- Application Number
- CN202410780708.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-06-18
AI Technical Summary
The existing video information extraction system cannot effectively extract the information in popular science videos, and does not perform video preprocessing, resulting in a large transmission volume and cannot meet the video needs for quick search.
Adopt a system based on edge computing and artificial intelligence analysis, including edge acquisition module, intelligent analysis module and storage management module, and process video frame images through edge computing, perform voice, image and text analysis, determine key information, and compress and store videos. , provides interactive application modules to quickly find videos.
Filter videos through edge computing, reduce transmission volume, reduce transmission burden, extract video feature information, and help users quickly find the video content they need.
Smart Images

Figure CN118368482B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electronic digital data processing, and in particular to a popular science video extraction system based on edge computing and artificial intelligence analysis. Background Art
[0002] Currently, learning through popular science videos on the Internet is a low-cost way of learning, and this learning method is becoming more and more popular. However, with the increase in the number and content of popular science videos, how to accurately find the required popular science videos has become a problem that needs to be solved.
[0003] Many video information extraction systems have been developed. After extensive searching and reference, we found that the existing information extraction systems include the system disclosed in the publication number CN107465911B. These system methods generally include: obtaining video frame image information, the video frame image information includes the first frame image information and the second frame image information; extracting edge contour information in the first frame image, and matching the first frame image information with the second frame image information to obtain sparse optical flow; calculating the dense optical flow between the first frame image information and the second frame image information based on the edge contour information and the sparse optical flow; converting the dense optical flow into initial depth information, and processing the initial depth information and the video frame image information to obtain the final depth information. However, this system is not suitable for extracting information from popular science videos, and the video is not preprocessed, so the transmission volume is large. Summary of the invention
[0004] The purpose of the present invention is to propose a popular science video extraction system based on edge computing and artificial intelligence analysis to address the existing deficiencies.
[0005] The present invention adopts the following technical solution:
[0006] A popular science video extraction system based on edge computing and artificial intelligence analysis, including an edge acquisition module, an intelligent analysis module, a storage management module and an interactive application module;
[0007] The edge acquisition module is used to collect original popular science videos and perform edge computing on the popular science videos. The intelligent analysis module is used to perform intelligent analysis on the voice, image and text of the videos after edge computing. The storage management module stores and manages the link addresses of the original popular science videos according to the analysis results. The interactive application module is used to provide an interactive interface and application services.
[0008] The edge acquisition module includes a video recognition unit, an edge processing unit and a video transmission unit. The video recognition unit is used to recognize the popular science video file, the edge processing unit is used to compress the popular science video file, and the video transmission unit is used to transmit the compressed video file to the intelligent analysis module;
[0009] The intelligent analysis module includes a voice analysis unit, an image analysis unit and a text analysis unit. The voice analysis unit is used to convert the voice information in the video into text information, the image analysis unit is used to analyze and process the image information in the video, and the text analysis unit is used to analyze and process the converted text information;
[0010] The storage management module includes a feature index unit and a storage unit, wherein the feature index unit obtains index features based on the analysis results, and the storage unit stores the link address of the original popular science video according to the index features;
[0011] Further, the edge processing unit includes a frame image processor and a compression information register, the frame image processor is used to analyze and process the frame image information and determine the frame information that needs to be retained, and the compression information register is used to store the retained frame information and audio stream information;
[0012] The process of the frame image processor processing the frame image information in the popular science video includes the following steps:
[0013] S1, take a frame of image as a reserved frame and send it to the compression information register;
[0014] S2, taking another frame image in sequence as the target frame, if there is no new frame image, then the process ends;
[0015] S3, graying the reserved frame and the target frame. If the reserved frame has been grayed, only the target frame needs to be grayed;
[0016] S4, subtracting the grayscale values of the corresponding pixels of the retained frame and the target frame to obtain a difference frame, wherein the value of each pixel in the difference frame is the absolute value of the grayscale value difference;
[0017] S5, dividing the difference frame into blocks, each block containing n*n pixels, and dividing the pixels into low difference pixels, medium difference pixels and high difference pixels according to their values;
[0018] S6. Calculate the difference index P of each block according to the following formula:
[0019] ;
[0020] Where n1 is the number of low-difference pixels, n2 is the number of medium-difference pixels, and n3 is the number of high-difference pixels;
[0021] S7. Calculate the change index Q of the entire difference frame according to the following formula:
[0022] ;
[0023] in, represents the difference index of the block in row x and column y;
[0024] When the change index Q is greater than the threshold, the target frame is sent to the compression information register as a new reserved frame;
[0025] S8, return to step S2;
[0026] Further, the image analysis unit includes a popular science content image conversion processor and a popular science content analysis processor, the popular science content image conversion processor is used to intercept the popular science content part from each frame of the image and convert the image content of the popular science part into first text information, and the popular science content analysis processor identifies and analyzes the first text information to obtain a popular science content data packet;
[0027] Further, the text information obtained by the text analysis unit from the speech analysis unit is called second text information, and the text analysis unit includes a time matching processor and a knowledge point confirmation processor, the time matching processor is used to divide the second text information into parts consistent with the number of the popular science content data packets, each part of the text information forms an annotation package, each annotation package is paired with a popular science content data packet, and the knowledge point confirmation processor obtains specific detailed knowledge points based on the text information in the annotation package and the knowledge point number analysis;
[0028] Furthermore, the knowledge point confirmation processor calculates the matching index R of each detailed knowledge point according to the following formula:
[0029] ;
[0030] in, represents the number of times the i-th keyword appears in the text information of the annotation package, and M is the number of keywords in a detailed knowledge point;
[0031] The detailed knowledge point with the largest matching index is used as the detailed knowledge point finally confirmed.
[0032] The beneficial effects achieved by the present invention are:
[0033] This system uses edge computing to filter the image frames of the original video, which can greatly reduce the transmission volume and reduce the transmission burden while retaining key information. At the same time, it performs image analysis and text analysis on the transmitted video. Graphic analysis narrows the scope of popular science content, and text analysis determines the detailed popular science content. Finally, it extracts feature information that can represent the popular science video, helping users to quickly find the video they need.
[0034] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;
[0036] Figure 2 This is a schematic diagram of the edge acquisition module of the present invention;
[0037] Figure 3 This is a schematic diagram of the intelligent analysis module of the present invention;
[0038] Figure 4 It is a schematic diagram of the edge processing unit of the present invention;
[0039] Figure 5 It is a schematic diagram of the image analysis unit of the present invention. DETAILED DESCRIPTION
[0040] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention. Example
[0041] This embodiment provides a popular science video extraction system based on edge computing and artificial intelligence analysis, combined with Figure 1 , including edge acquisition module, intelligent analysis module, storage management module and interactive application module;
[0042] The edge acquisition module is used to collect original popular science videos and perform edge computing on the popular science videos. The intelligent analysis module is used to perform intelligent analysis on the voice, image and text of the videos after edge computing. The storage management module stores and manages the link addresses of the original popular science videos according to the analysis results. The interactive application module is used to provide an interactive interface and application services.
[0043] The edge acquisition module includes a video recognition unit, an edge processing unit and a video transmission unit. The video recognition unit is used to recognize the popular science video file, the edge processing unit is used to compress the popular science video file, and the video transmission unit is used to transmit the compressed video file to the intelligent analysis module;
[0044] The intelligent analysis module includes a voice analysis unit, an image analysis unit and a text analysis unit. The voice analysis unit is used to convert the voice information in the video into text information, the image analysis unit is used to analyze and process the image information in the video, and the text analysis unit is used to analyze and process the converted text information;
[0045] The storage management module includes a feature index unit and a storage unit, wherein the feature index unit obtains index features based on the analysis results, and the storage unit stores the link address of the original popular science video according to the index features;
[0046] The edge processing unit includes a frame image processor and a compression information register, wherein the frame image processor is used to analyze and process the frame image information and determine the frame information to be retained, and the compression information register is used to store the retained frame information and audio stream information;
[0047] The process of the frame image processor processing the frame image information in the popular science video includes the following steps:
[0048] S1, take a frame of image as a reserved frame and send it to the compression information register;
[0049] S2, taking another frame image in sequence as the target frame, if there is no new frame image, then the process ends;
[0050] S3, graying the reserved frame and the target frame. If the reserved frame has been grayed, only the target frame needs to be grayed;
[0051] S4, subtracting the grayscale values of the corresponding pixels of the retained frame and the target frame to obtain a difference frame, wherein the value of each pixel in the difference frame is the absolute value of the grayscale value difference;
[0052] S5, dividing the difference frame into blocks, each block containing n*n pixels, and dividing the pixels into low difference pixels, medium difference pixels and high difference pixels according to their values;
[0053] S6. Calculate the difference index P of each block according to the following formula:
[0054] ;
[0055] Where n1 is the number of low-difference pixels, n2 is the number of medium-difference pixels, and n3 is the number of high-difference pixels;
[0056] S7. Calculate the change index Q of the entire difference frame according to the following formula:
[0057] ;
[0058] in, represents the difference index of the block in row x and column y;
[0059] When the change index Q is greater than the threshold, the target frame is sent to the compression information register as a new reserved frame;
[0060] S8, return to step S2;
[0061] The image analysis unit includes a popular science content image conversion processor and a popular science content analysis processor, the popular science content image conversion processor is used to intercept the popular science content part from each frame of the image and convert the image content of the popular science part into first text information, and the popular science content analysis processor identifies and analyzes the first text information to obtain a popular science content data packet;
[0062] The text information obtained by the text analysis unit from the speech analysis unit is called second text information. The text analysis unit includes a time matching processor and a knowledge point confirmation processor. The time matching processor is used to divide the second text information into parts that are consistent with the number of the popular science content data packets, each part of the text information forms an annotation package, and each annotation package is paired with a popular science content data packet. The knowledge point confirmation processor obtains specific detailed knowledge points based on the text information in the annotation package and the knowledge point number analysis;
[0063] The knowledge point confirmation processor calculates the matching index R of each detailed knowledge point according to the following formula:
[0064] ;
[0065] in, represents the number of times the i-th keyword appears in the text information of the annotation package, and M is the number of keywords in a detailed knowledge point;
[0066] The detailed knowledge point with the largest matching index is used as the detailed knowledge point finally confirmed. Example
[0067] This embodiment includes all the contents of the first embodiment, and provides a popular science video extraction system based on edge computing and artificial intelligence analysis, including an edge acquisition module, an intelligent analysis module, a storage management module, and an interactive application module;
[0068] The edge acquisition module is used to collect original popular science videos and perform edge computing on the popular science videos. The intelligent analysis module is used to perform intelligent analysis on the voice, image and text of the videos after edge computing. The storage management module stores and manages the link addresses of the original popular science videos according to the analysis results. The interactive application module is used to provide an interactive interface and application services.
[0069] Combination Figure 2 The edge acquisition module includes a video recognition unit, an edge processing unit and a video transmission unit. The video recognition unit is used to recognize the popular science video file, the edge processing unit is used to compress the popular science video file, and the video transmission unit is used to transmit the compressed video file to the intelligent analysis module;
[0070] Combination Figure 3 The intelligent analysis module includes a voice analysis unit, an image analysis unit and a text analysis unit. The voice analysis unit is used to convert the voice information in the video into text information, the image analysis unit is used to analyze and process the image information in the video, and the text analysis unit is used to analyze and process the converted text information;
[0071] The storage management module includes a feature index unit and a storage unit, wherein the feature index unit obtains index features based on the analysis results, and the storage unit stores the link address of the original popular science video according to the index features;
[0072] The interactive application module includes a display unit, an input unit and an application processing unit, wherein the display unit is used to display interactive content, the input unit is used to input required information, and the application processing unit processes the input information and obtains the corresponding link address from the storage management module;
[0073] The video recognition unit includes a format recognition processor, a title recognition processor and an address recognition processor. The format recognition processor is used to recognize the file format and send the title information of the video file to the title recognition processor. The title recognition processor analyzes the title information to determine whether it is a popular science video. The address recognition processor obtains the link address information of the popular science video.
[0074] The title recognition processor is provided with a popular science keyword, and when the popular science keyword is detected in the title information, the video is determined to be a popular science video;
[0075] Combination Figure 4 , the edge processing unit includes a frame image processor and a compression information register, the frame image processor is used to analyze and process the frame image information and determine the frame information that needs to be retained, and the compression information register is used to save the retained frame information and audio stream information;
[0076] The process of the frame image processor processing the frame image information in the popular science video includes the following steps:
[0077] S1, take a frame of image as a reserved frame and send it to the compression information register;
[0078] S2, taking another frame image in sequence as the target frame, if there is no new frame image, then the process ends;
[0079] S3, graying the reserved frame and the target frame. If the reserved frame has been grayed, only the target frame needs to be grayed;
[0080] S4, subtracting the grayscale values of the corresponding pixels of the retained frame and the target frame to obtain a difference frame, wherein the value of each pixel in the difference frame is the absolute value of the grayscale value difference;
[0081] S5, dividing the difference frame into blocks, each block containing n*n pixels, and dividing the pixels into low difference pixels, medium difference pixels and high difference pixels according to their values;
[0082] S6. Calculate the difference index P of each block according to the following formula:
[0083] ;
[0084] Where n1 is the number of low-difference pixels, n2 is the number of medium-difference pixels, and n3 is the number of high-difference pixels;
[0085] S7. Calculate the change index Q of the entire difference frame according to the following formula:
[0086] ;
[0087] in, represents the difference index of the block in row x and column y;
[0088] When the change index Q is greater than the threshold, the target frame is sent to the compression information register as a new reserved frame;
[0089] S8, return to step S2;
[0090] The video transmission unit obtains content from the compression information register and the address recognition processor and sends it to the intelligent analysis module;
[0091] Combination Figure 5 The image analysis unit includes a popular science content image conversion processor and a popular science content analysis processor, the popular science content image conversion processor is used to intercept the popular science content part from each frame of the image and convert the image content of the popular science part into first text information, and the popular science content analysis processor performs recognition and analysis processing on the first text information;
[0092] The processing process of the popular science content image conversion processor includes the following steps:
[0093] S21, determining a popular science content area according to a plurality of continuous image frames, and acquiring a first frame of image;
[0094] S22, intercepting image information in the popular science content area in the image frame;
[0095] S23, obtaining recognizable content in the image information and converting it into text information;
[0096] S24, comparing the newly obtained text information with the reference information, if there is new content information, adding the new content information to the reference information, if the newly obtained text information is all new content information, packaging the original reference information into a popular science content data packet and sending it to the popular science content processor, using the newly obtained text information as new reference information, if the comparison results are consistent, no processing is performed;
[0097] S25, obtaining the next frame of image, and returning to step S22;
[0098] The popular science content analysis processor is provided with knowledge point data, the knowledge point data including knowledge point type, knowledge point number and knowledge point feature, the knowledge point type includes subject types such as Chinese, mathematics, English, etc., the knowledge point number is used to indicate the knowledge point range corresponding to the popular science content, and the knowledge point feature is exclusive information used to identify the knowledge point;
[0099] The popular science content analysis processor determines the type of knowledge point according to the ratio of numerical formulas, English words and characters appearing in the text information. Each knowledge point type has a preset ratio interval. The popular science content analysis processor compares the exclusive information under the corresponding type with the text information to determine the knowledge point number.
[0100] The text information obtained by the text analysis unit from the speech analysis unit is called second text information. The text analysis unit includes a time matching processor and a knowledge point confirmation processor. The time matching processor is used to divide the second text information into parts that are consistent with the number of the popular science content data packets, each part of the text information forms an annotation package, and each annotation package is paired with a popular science content data packet. The pairing is based on the time axis information of the image frame and the time axis information of the second text information processed by the speech analysis unit. The knowledge point confirmation processor obtains specific detailed knowledge points based on the text information in the annotation package and the knowledge point number analysis;
[0101] The knowledge point range corresponding to each knowledge point number includes at least two detailed knowledge points, and each detailed knowledge point is provided with keyword information. The knowledge point confirmation processor obtains the keyword information of all corresponding detailed knowledge points based on the knowledge point number, compares the keyword information with the text information, and then determines the detailed knowledge point according to the following formula:
[0102] ;
[0103] in, represents the number of times the i-th keyword appears in the text information, M is the number of keywords in a detailed knowledge point, and R is the matching index of the detailed knowledge point;
[0104] The detailed knowledge point with the largest matching index is used as the final confirmed detailed knowledge point;
[0105] The feature index unit includes a detailed knowledge point merging processor and an index feature processor. The detailed knowledge point merging processor is used to count the corresponding time length of each detailed knowledge point and add the time lengths of the same detailed knowledge points. The index feature processor calculates the corresponding index features based on the distribution and time length of the detailed knowledge points contained in the learning video. The index features include the main detailed knowledge point, the number of detailed knowledge points included, and the duration ratio of the main detailed knowledge point, wherein the main detailed knowledge point is the detailed knowledge point with the largest time length.
[0106] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.
Claims
1. A popular science video extraction system based on edge computing and artificial intelligence analysis, characterized in that: It includes edge acquisition module, intelligent analysis module, storage management module and interactive application module; The edge acquisition module is used to collect original popular science videos and perform edge computing on the popular science videos. The intelligent analysis module is used to perform intelligent analysis on the voice, image and text of the videos after edge computing. The storage management module stores and manages the link addresses of the original popular science videos according to the analysis results. The interactive application module is used to provide an interactive interface and application services. The edge acquisition module includes a video recognition unit, an edge processing unit and a video transmission unit. The video recognition unit is used to recognize the popular science video file, the edge processing unit is used to compress the popular science video file, and the video transmission unit is used to transmit the compressed video file to the intelligent analysis module; The intelligent analysis module includes a voice analysis unit, an image analysis unit and a text analysis unit. The voice analysis unit is used to convert the voice information in the video into text information, the image analysis unit is used to analyze and process the image information in the video, and the text analysis unit is used to analyze and process the converted text information; The storage management module includes a feature index unit and a storage unit, wherein the feature index unit obtains an index feature based on the analysis result, and the storage unit stores the link address of the original popular science video according to the index feature; the edge processing unit includes a frame image processor and a compression information register, wherein the frame image processor is used to analyze and process the frame image information and determine the frame information to be retained, and the compression information register is used to save the retained frame information and audio stream information; The process of the frame image processor processing the frame image information in the popular science video includes the following steps: S1, take a frame of image as a reserved frame and send it to the compression information register; S2, taking another frame image in sequence as the target frame, and proceeding to step S3, if there is no new frame image, the process ends; S3, graying the reserved frame and the target frame. If the reserved frame has been grayed, only the target frame needs to be grayed; S4, subtracting the grayscale values of the corresponding pixels of the retained frame and the target frame to obtain a difference frame, wherein the value of each pixel in the difference frame is the absolute value of the grayscale value difference; S5, dividing the difference frame into blocks, each block containing n*n pixels, and dividing the pixels into low difference pixels, medium difference pixels and high difference pixels according to their values; S6. Calculate the difference index P of each block according to the following formula: ; Where n1 is the number of low-difference pixels, n2 is the number of medium-difference pixels, and n3 is the number of high-difference pixels; S7. Calculate the change index Q of the entire difference frame according to the following formula: ; in, represents the difference index of the block in row x and column y; When the change index Q is greater than the threshold, the target frame is sent to the compression information register as a new reserved frame; S8. Return to step S2.
2. A popular science video extraction system based on edge computing and artificial intelligence analysis as claimed in claim 1, characterized in that: The image analysis unit includes a popular science content image conversion processor and a popular science content analysis processor. The popular science content image conversion processor is used to intercept the popular science content part from each frame of the image and convert the image content of the popular science part into first text information. The popular science content analysis processor identifies and analyzes the first text information to obtain a popular science content data packet.
3. A popular science video extraction system based on edge computing and artificial intelligence analysis as claimed in claim 2, characterized in that: The text information obtained by the text analysis unit from the speech analysis unit is called the second text information. The text analysis unit includes a time matching processor and a knowledge point confirmation processor. The time matching processor is used to divide the second text information into parts consistent with the number of popular science content data packets, each part of the text information forms an annotation package, and each annotation package is paired with a popular science content data packet. The knowledge point confirmation processor obtains specific detailed knowledge points based on the text information in the annotation package and the knowledge point number analysis.
4. A popular science video extraction system based on edge computing and artificial intelligence analysis as claimed in claim 3, characterized in that: The knowledge point confirmation processor calculates the matching index R of each detailed knowledge point according to the following formula: ; in, represents the number of times the i-th keyword appears in the text information of the annotation package, and M is the number of keywords in a detailed knowledge point; The detailed knowledge point with the largest matching index is used as the detailed knowledge point finally confirmed.
