A method and system for intelligent video analysis based on AI

Abstract

The present invention relates to the field of video analysis technology, and specifically discloses a method and system for intelligent video analysis based on AI, wherein the method comprises segmenting the video to be identified to obtain audio information and image sequences based on the same timeline; identifying the audio information, performing initial grouping of the image sequence, performing intra-group comparison on each group of images after the initial grouping, and performing secondary grouping according to the comparison results to obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on the timeline; extracting at least one image from each image group, identifying the image, and generating an evaluation text. The present invention extracts some images from a video file, and analyzes the images with the help of a fixed recognition scheme based on AI and a dynamic recognition scheme based on a local model, so that the video can be converted into text very quickly, which is convenient for users to quickly understand the video content.

Description

Technical Field

[0001] The present invention relates to the field of video analysis technology, and specifically to an AI-based intelligent video analysis method and system. Background Art

[0002] Video refers to various technologies that capture, record, process, store, transmit and reproduce a series of static images in the form of electrical signals. When the continuous image changes more than 24 frames per second, according to the principle of visual persistence, the human eye cannot distinguish a single static picture; it looks like a smooth and continuous visual effect, and such a continuous picture is called video.

[0003] The performance of existing popular computer equipment is very high, which makes video gradually become mainstream media data, but the volume is large, and it is difficult for readers to get a preliminary understanding of the video content in advance. They can only read the introduction. For some videos without introduction or very brief introduction, readers can only check it by themselves. In situations where video screening is required, the screening process is very difficult. If a video content analysis solution can be provided to enable readers to understand the video content in advance, it can greatly simplify the screening difficulty and improve the screening efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide an AI-based intelligent video analysis method and system to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An AI-based intelligent video analysis method, the method comprising:

[0007] Open the video receiving port to receive the video to be identified;

[0008] Segmenting the video to be identified to obtain audio information and image sequences based on the same time axis;

[0009] The audio information is identified, the image sequence is initially grouped, each group of images after the initial grouping is compared within the group, and a secondary grouping is performed according to the comparison result to obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on a time axis;

[0010] Extract at least one image from each image group, identify the image, and generate evaluation text;

[0011] The process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, wherein the dynamic recognition scheme is a recognition scheme based on a neural network model.

[0012] As a further solution of the present invention: the step of segmenting the video to be identified to obtain audio information and image sequences based on the same time axis includes:

[0013] Get the time span of the video to be identified and create a timeline;

[0014] Extracting audio information from the audio track of the video to be identified based on the timeline;

[0015] Obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points;

[0016] Based on the sorting order of the image points, the corresponding images are counted to obtain an image sequence.

[0017] As a further solution of the present invention: the steps of identifying the audio information, performing initial grouping of the image sequence, performing intra-group comparison on each group of images after the initial grouping, and performing secondary grouping according to the comparison results to obtain the final image group include:

[0018] Reading the amplitude of the audio information in time sequence, and segmenting the audio information based on the amplitude;

[0019] Read all images corresponding to each audio information segment, retain the image order, and obtain the initial grouping result;

[0020] In each group of images after the initial grouping, the images are sequentially compared with a preset number of adjacent images to determine the prevalence of the images;

[0021] The same group of images is grouped again according to the prevalence to obtain a final image group.

[0022] As a further solution of the present invention: the process of segmenting the audio information is as follows:

[0023] Read the audio amplitude according to the time sequence, and generate segmentation points according to the audio amplitude;

[0024] ; Indicates segmentation points, express The previous segmentation point, Indicates the time difference, , Indicates duration The standard deviation of the amplitude within the range, is the preset threshold; Indicates that it meets the conditions The maximum value of ;

[0025] The process of determining the prevalence of an image is:

[0026] ; represents the prevalence of the image, Indicates the preset image selection radius, which is a positive value; Indicates the first The similarity between the previous image and the current image, the similarity value ranges from zero to one; is the preset adjustment constant.

[0027] As a further solution of the present invention: the step of extracting at least one image from each image group, identifying the image, and generating an evaluation text comprises:

[0028] For each image group, read the prevalence of each image, and extract at least one image according to the prevalence;

[0029] Perform subject recognition on the extracted images to obtain subject phrases;

[0030] Arranging the subject phrases of each image according to the image order of the image sequence, and converting the subject phrases into sentences based on a statistical language model;

[0031] Count all the sentences and get the evaluation text.

[0032] As a further solution of the present invention: the step of extracting at least one image from each image group, identifying the image, and generating an evaluation text further includes:

[0033] Pre-constructing a sample set of images to description sentences, and training a neural network model based on the sample set;

[0034] When an image is recognized, the image and its corresponding sentence are used as new samples to update the sample set;

[0035] Determine the error rate of the neural network model in real time according to the new sample, and adjust the application frequency of the fixed recognition scheme according to the error rate;

[0036] Send new samples to the verification end regularly and receive sample correction instructions from the verification end.

[0037] The technical solution of the present invention also provides an AI-based intelligent video analysis system, the system comprising:

[0038] A video receiving module, used for opening a video receiving port and receiving a video to be identified;

[0039] A component extraction module, used for segmenting the video to be identified to obtain audio information and image sequences based on the same time axis;

[0040] A video grouping module is used to identify the audio information, perform initial grouping on the image sequence, perform intra-group comparison on each group of images after the initial grouping, perform secondary grouping based on the comparison results, and obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on the time axis;

[0041] An image recognition module is used to extract at least one image from each image group, recognize the image, and generate an evaluation text;

[0042] The process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, wherein the dynamic recognition scheme is a recognition scheme based on a neural network model.

[0043] As a further solution of the present invention: the component extraction module includes:

[0044] A timeline creation unit, used to obtain the time span of the video to be identified and create a timeline;

[0045] An audio information extraction unit, used for extracting audio information from an audio track of a video to be identified based on a time axis;

[0046] An image insertion unit, used to obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points;

[0047] The image arrangement unit is used to count the corresponding images based on the sorting order of the image points to obtain an image sequence.

[0048] As a further solution of the present invention: the video grouping module includes:

[0049] An audio information analysis unit, configured to read the amplitude of the audio information according to a time sequence, and segment the audio information based on the amplitude;

[0050] An image reading unit, used for reading all images corresponding to each audio information segment, preserving the image sequence, and obtaining a primary grouping result;

[0051] a prevalence calculation unit, for comparing the image with a preset number of adjacent images in each group of images after the initial grouping, to determine the prevalence of the image;

[0052] The secondary grouping unit is used to perform secondary grouping on the same group of images according to the prevalence to obtain a final image group.

[0053] As a further solution of the present invention: the image recognition module includes:

[0054] An extraction execution unit, configured to read the prevalence of each image in each image group, and extract at least one image according to the prevalence;

[0055] A subject recognition unit is used to perform subject recognition on the extracted image to obtain a subject phrase;

[0056] a sentence generation unit, for arranging the subject phrases of each image according to the image order of the image sequence, and converting the subject phrases into sentences based on a statistical language model;

[0057] The sentence statistics unit is used to count all sentences and obtain evaluation texts.

[0058] Compared with the prior art, the beneficial effects of the present invention are: the present invention extracts part of the images from the video file, analyzes the images with the help of a fixed recognition scheme based on AI and a dynamic recognition scheme based on a local model, and trains the local model on the basis of generating the analysis content, and finally obtains an extremely efficient local model. When receiving an analysis request, the video can be converted into text very quickly, so that users can quickly understand the video content. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0060] Figure 1 This is a flowchart of the AI-based intelligent video analysis method.

[0061] Figure 2 This is a structural block diagram of the AI-based intelligent video analysis system. DETAILED DESCRIPTION

[0062] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] Figure 1 The flowchart of the AI-based intelligent video analysis method is as follows. In an embodiment of the present invention, an AI-based intelligent video analysis method includes:

[0064] Step S100: Open the video receiving port to receive the video to be identified;

[0065] The data that needs to be processed by the technical solution of the present invention is video. Therefore, it is necessary to obtain the video first and then process it. The obtained video is called the video to be identified; the port used to obtain the video is called the video receiving port, which needs to be connected to videos in various formats, and then converted into video data in a preset format for subsequent processing.

[0066] Step S200: segmenting the video to be identified to obtain audio information and image sequences based on the same time axis;

[0067] A video file mainly consists of two parts, one is the audio part and the other is the image part. The process of dividing the video to be identified into the audio part and the image part is not complicated. It should be noted that the split audio part and the image part share the same timeline.

[0068] Step S300: Identify the audio information, perform initial grouping on the image sequence, perform intra-group comparison on each group of images after the initial grouping, perform secondary grouping based on the comparison results, and obtain a final image group; the image group is a collection of images, and the image group contains a time tag generated based on the time axis;

[0069] By identifying the audio information, the images in the image sequence can be preliminarily grouped. This grouping is a simple grouping process. For example, the sound segments are grouped together and the silent segments are grouped together. On this basis, the divided image groups themselves are further identified and secondary grouping is performed. Since the secondary grouping directly processes the image itself, the obtained grouping result is more accurate.

[0070] The reason for setting a step-by-step grouping scheme in the above content is to reduce the number of image comparisons. Since the image comparison process consumes more resources, after pre-grouping based on audio information, the number of images that need to be compared can be effectively reduced, thereby reducing resource consumption.

[0071] It should be noted that another reason for reducing resource consumption is that after the images are grouped according to the audio information, the images in the same group themselves have certain similarities. At least the video content is likely to be content occurring in the same scene, and the images themselves are similar. In existing processing equipment, it is often necessary to pre-load some parameters (preprocessing process before image processing). On the basis that the images are relatively similar, the preprocessing process hardly needs to be changed, thereby reducing resource consumption.

[0072] Step S400: extract at least one image from each image group, identify the image, and generate evaluation text;

[0073] After the image grouping is completed, at least one image is extracted from each image group, the image is identified, and an evaluation text is generated. The popular meaning of the evaluation text is the language description of the image, and then the video is converted into a text description, which makes it easier for readers to quickly understand the entire video content.

[0074] It should be noted that in step S400, the process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, and the dynamic recognition scheme is a recognition scheme based on a neural network model. In the technical solution of the present invention, the fixed recognition scheme involved is a scheme that uses an existing image to text conversion. The simplest and most direct way is to use AI. The existing AI can recognize images and generate text. On this basis, the present application trains a local neural network model based on the recognition results, that is, the local neural network model is trained with the results of a large AI model that requires a network. When the training is completed, while ensuring a certain degree of accuracy, the network interaction time is saved, greatly improving the recognition efficiency.

[0075] Regarding step S200, the step of segmenting the video to be identified to obtain audio information and image sequences based on the same time axis includes:

[0076] Get the time span of the video to be identified and create a timeline;

[0077] Extracting audio information from the audio track of the video to be identified based on the timeline;

[0078] Obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points;

[0079] Based on the sorting order of the image points, the corresponding images are counted to obtain an image sequence.

[0080] The above content is related to the component segmentation of the video. The time span of the video to be identified is obtained, and a timeline is created. The initial time of the timeline is time zero, and the end time is the length of the time span of the video; audio information is extracted from the audio track of the video to be identified, and the time tag of the audio information is recorded and transferred to the timeline; then, the frame rate of the video is obtained. The frame rate indicates how many images there are in one second. Based on the number of images, each second is divided to obtain multiple time points, which are called image points. The image is read on the image track of the video to be identified and matched with each image point.

[0081] Regarding step S300, the steps of identifying the audio information, performing initial grouping on the image sequence, performing intra-group comparison on each group of images after the initial grouping, and performing secondary grouping according to the comparison results to obtain the final image group include:

[0082] Reading the amplitude of the audio information in time sequence, and segmenting the audio information based on the amplitude;

[0083] Read all images corresponding to each audio information segment, retain the image order, and obtain the initial grouping result;

[0084] In each group of images after the initial grouping, the images are sequentially compared with a preset number of adjacent images to determine the prevalence of the images;

[0085] The same group of images is grouped again according to the prevalence to obtain a final image group.

[0086] The above content describes the image grouping process. The amplitude of the audio information is obtained in sequence according to the time sequence (the vertical coordinates are queried in sequence in the audio time domain diagram), and the amplitudes are compared. The audio information can be preliminarily segmented, and all images corresponding to each segment of audio information are read, and the image order is retained to obtain the initial grouping result; in each group of images after the initial grouping, the images are taken in turn, and the images are compared with a preset number of adjacent images to determine the prevalence of the images. Generally, the comparison process occurs in an image and several images around it. The inverse ratio of the difference between the image and the surrounding images is called prevalence; the initial grouping results are grouped according to the prevalence to obtain the final image group.

[0087] Specifically, the process of segmenting the audio information is as follows:

[0088] Read the audio amplitude according to the time sequence, and generate segmentation points according to the audio amplitude;

[0089] ; Indicates segmentation points, express The previous segmentation point, represents the initial time, The maximum value of is a positive integer, Indicates the time difference, , Indicates duration The standard deviation of the amplitude within the range, is the preset threshold; Indicates that it meets the conditions The maximum value of ;

[0090] The process of determining the prevalence of an image is:

[0091] ; represents the prevalence of the image, Indicates the preset image selection radius, which is a positive value; Indicates the first The similarity between the previous image and the current image, the similarity value ranges from zero to one. is the preset adjustment constant.

[0092] In an example of the technical solution of the present invention, the segmentation process of audio information and the calculation process of image prevalence, the principle of the segmentation process of audio information is to first take the initial moment as the starting point and calculate a time difference. The meaning of the time difference is that as time goes by, the audio segments gradually increase, and the standard deviation of the amplitude of the audio segments is calculated. If the amplitudes of the audio segments are similar, the standard deviation will not be large. If the standard deviation is large, it means that the audio segment has a mutation. When the standard deviation is large enough, the time difference is recorded. On the basis of the previous moment, a time difference is added to get the next moment. All selected time points are segmentation points, and the time axis is divided into multiple segments to realize the segmentation function.

[0093] Regarding the universality of an image, its principle is very simple, that is, it is proportional to the similarity between the image and the surrounding images.

[0094] Regarding step S400, the step of extracting at least one image from each image group, identifying the image, and generating an evaluation text includes:

[0095] For each image group, read the prevalence of each image, and extract at least one image according to the prevalence;

[0096] Perform subject recognition on the extracted images to obtain subject phrases;

[0097] Arranging the subject phrases of each image according to the image order of the image sequence, and converting the subject phrases into sentences based on a statistical language model;

[0098] Count all the sentences and get the evaluation text.

[0099] The above content describes the image recognition process in detail. After the images are completely grouped, the output is an image group. Each image group needs to be analyzed separately. For each image group, the prevalence of each image is read, and images are selected based on the prevalence as images to be recognized. This process is further simplified. Since all images in the grouped image groups are similar in themselves, only a limited number of images are selected from each image group for recognition.

[0100] The solution provided in the above content is actually a fixed recognition solution. In the technical solution of the present invention, a fixed recognition solution based on AI has been provided. However, AI requires high computing resources and the processing speed is not very fast. Therefore, this application also provides some other fixed recognition solutions. The subject recognition algorithm is used to perform subject recognition on the image (existing recognition algorithm). After the subject recognition of the image, the output obtained is the name of each subject, which is called the subject phrase; the image sequence in the image sequence is queried, and the subject phrase is obtained according to the sequence arrangement. The subject phrase is converted into a sentence based on the statistical language model. The core idea of ​​the statistical language model is to convert the language problem into a probability calculation problem. It determines the most likely sentence structure or text classification by calculating the probability distribution of words in the text. The statistical language model usually adopts a probabilistic graph model, such as a hidden Markov model (HMM) and a conditional random field (CRF), etc., to describe the relationship between words, and uses probability distribution to represent the probability of words. With the help of the statistical language model, words can be converted into sentences, and the converted sentences are the text description of the image, that is, the evaluation text.

[0101] It should be noted that, in the process of extracting at least one image according to the prevalence, the selection probability of each image is inversely proportional to the prevalence of the image, that is, the more prevalent the image is, the less likely it is to be selected; in addition, when more than one image is selected, the interference of adjacent images can also be introduced, as follows:

[0102] ;in, represents the probability of selecting a certain image, Indicates the prevalence of the image. Indicates the total number of other selected images in the image group. Indicates the other selected images in the image group. The number of images between the previous image and the current image. Since this parameter is very similar to the concept of distance, it can also be called image distance.

[0103] As a preferred embodiment of the technical solution of the present invention, the step of extracting at least one image from each image group, identifying the image, and generating an evaluation text further includes:

[0104] Pre-constructing a sample set of images to description sentences, and training a neural network model based on the sample set;

[0105] When an image is recognized, the image and its corresponding sentence are used as new samples to update the sample set;

[0106] Determine the error rate of the neural network model in real time according to the new sample, and adjust the application frequency of the fixed recognition scheme according to the error rate;

[0107] Send new samples to the verification end regularly and receive sample correction instructions from the verification end.

[0108] The above content describes a dynamic recognition scheme. The management party inputs some images and their description sentences in advance as the initial sample set. The neural network model is trained with the help of the initial sample set to obtain the mapping relationship between the image and the description sentence. Since the number of samples in the initial sample set is small, the error rate of the trained neural network model is very high. On this basis, when a fixed recognition scheme is used to recognize an image, the image and its corresponding fixed recognition result can be used as a new sample to update the sample set. While the fixed recognition scheme is used to recognize the image, the image is input into the neural network model to obtain the output of the neural network model. The output is compared with the result of the fixed recognition scheme, and the error rate can be calculated (the fixed recognition scheme is used to identify the image). The output of the recognition scheme is taken as the true value, and the output of the neural network model is taken as the predicted value), and finally, the application frequency of the fixed recognition scheme is adjusted by the error rate; the application frequency of the fixed recognition scheme is proportional to the error rate. The smaller the error rate, the more accurate the output of the neural network model. Correspondingly, the application frequency of the fixed recognition scheme should be smaller. In the extreme case, the error rate is zero, and the application frequency of the fixed recognition scheme is also zero. At this time, the recognition process is completed entirely by the neural network model; the practical significance of this result is that after the neural network model is trained, the trained neural network model replaces the fixed recognition scheme. The trained neural network model is a local model with an extremely fast response speed, which is much higher than the fixed recognition scheme.

[0109] Figure 2 The structure diagram of the AI-based intelligent video analysis system is shown in FIG. 1 . In an embodiment of the present invention, an AI-based intelligent video analysis system 10 includes:

[0110] The video receiving module 11 is used to open a video receiving port and receive a video to be identified;

[0111] A component extraction module 12 is used to segment the video to be identified to obtain audio information and image sequences based on the same time axis;

[0112] The video grouping module 13 is used to identify the audio information, perform initial grouping on the image sequence, perform intra-group comparison on each group of images after the initial grouping, perform secondary grouping based on the comparison results, and obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on the time axis;

[0113] An image recognition module 14 is used to extract at least one image from each image group, recognize the image, and generate an evaluation text;

[0114] The process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, wherein the dynamic recognition scheme is a recognition scheme based on a neural network model.

[0115] Furthermore, the component extraction module 12 includes:

[0116] A timeline creation unit, used to obtain the time span of the video to be identified and create a timeline;

[0117] An audio information extraction unit, used for extracting audio information from an audio track of a video to be identified based on a time axis;

[0118] An image insertion unit, used to obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points;

[0119] The image arrangement unit is used to count the corresponding images based on the sorting order of the image points to obtain an image sequence.

[0120] Specifically, the video grouping module 13 includes:

[0121] An audio information analysis unit, configured to read the amplitude of the audio information according to a time sequence, and segment the audio information based on the amplitude;

[0122] An image reading unit, used for reading all images corresponding to each audio information segment, preserving the image sequence, and obtaining a primary grouping result;

[0123] a prevalence calculation unit, for comparing the image with a preset number of adjacent images in each group of images after the initial grouping, to determine the prevalence of the image;

[0124] The secondary grouping unit is used to perform secondary grouping on the same group of images according to the prevalence to obtain a final image group.

[0125] Furthermore, the image recognition module 14 includes:

[0126] An extraction execution unit, configured to read the prevalence of each image in each image group, and extract at least one image according to the prevalence;

[0127] A subject recognition unit is used to perform subject recognition on the extracted image to obtain a subject phrase;

[0128] a sentence generation unit, for arranging the subject phrases of each image according to the image order of the image sequence, and converting the subject phrases into sentences based on a statistical language model;

[0129] The sentence statistics unit is used to count all sentences and obtain evaluation texts.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for analyzing video based on AI, characterized in that: The method comprises: Open the video receiving port to receive the video to be identified; Segmenting the video to be identified to obtain audio information and image sequences based on the same time axis; The audio information is identified, the image sequence is initially grouped, each group of images after the initial grouping is compared within the group, and a secondary grouping is performed according to the comparison result to obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on a time axis; Extract at least one image from each image group, identify the image, and generate evaluation text; The process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, wherein the dynamic recognition scheme is a recognition scheme based on a neural network model; The steps of identifying the audio information, performing initial grouping on the image sequence, performing intra-group comparison on each group of images after the initial grouping, and performing secondary grouping according to the comparison results to obtain the final image group include: Reading the amplitude of the audio information in time sequence, and segmenting the audio information based on the amplitude; Read all images corresponding to each audio information segment, retain the image order, and obtain the initial grouping result; In each group of images after the initial grouping, the images are sequentially compared with a preset number of adjacent images to determine the prevalence of the images; Performing secondary grouping on the same group of images according to the prevalence to obtain a final group of images; The step of extracting at least one image from each image group, identifying the image, and generating an evaluation text comprises: For each image group, read the prevalence of each image, and extract at least one image according to the prevalence; Perform subject recognition on the extracted images to obtain subject phrases; Arrange the subject phrases of each image according to the image order of the image sequence, and convert the subject phrases into sentences based on the statistical language model; Count all the sentences and get the evaluation text; The step of extracting at least one image from each image group, identifying the image, and generating an evaluation text also includes: Pre-constructing a sample set of images to description sentences, and training a neural network model based on the sample set; When an image is recognized, the image and its corresponding sentence are used as new samples to update the sample set; Determine the error rate of the neural network model in real time according to the new sample, and adjust the application frequency of the fixed recognition scheme according to the error rate; Send new samples to the verification end regularly and receive sample correction instructions from the verification end.

2. The AI-based intelligent video analysis method according to claim 1 is characterized in that: The step of segmenting the video to be identified to obtain audio information and image sequences based on the same time axis includes: Get the time span of the video to be identified and create a timeline; Extracting audio information from the audio track of the video to be identified based on the timeline; Obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points; Based on the sorting order of the image points, the corresponding images are counted to obtain an image sequence.

3. The AI-based intelligent video analysis method according to claim 1 is characterized in that: The process of segmenting audio information is: Read the audio amplitude according to the time sequence, and generate segmentation points according to the audio amplitude; ; Indicates segmentation points, express The previous segmentation point, Indicates the time difference, , Indicates duration The standard deviation of the amplitude within the range, is the preset threshold; Indicates that it meets the conditions The maximum value of ; The process of determining the prevalence of an image is: ; represents the prevalence of the image, Indicates the preset image selection radius, which is a positive value; Indicates the first The similarity between the previous image and the current image, the similarity value ranges from zero to one; is the preset adjustment constant.

4. An AI-based intelligent video analysis system, characterized in that: The system comprises: A video receiving module, used for opening a video receiving port and receiving a video to be identified; A component extraction module, used for segmenting the video to be identified to obtain audio information and image sequences based on the same time axis; A video grouping module is used to identify the audio information, perform initial grouping on the image sequence, perform intra-group comparison on each group of images after the initial grouping, perform secondary grouping based on the comparison results, and obtain a final image group; the image group is a collection of images, and the image group contains a time label generated based on the time axis; An image recognition module is used to extract at least one image from each image group, recognize the image, and generate an evaluation text; The process of recognizing an image includes a fixed recognition scheme and a dynamic recognition scheme, wherein the dynamic recognition scheme is a recognition scheme based on a neural network model; The video grouping module comprises: An audio information analysis unit, configured to read the amplitude of the audio information according to a time sequence, and segment the audio information based on the amplitude; An image reading unit, used for reading all images corresponding to each audio information segment, preserving the image sequence, and obtaining a primary grouping result; a prevalence calculation unit, for comparing the image with a preset number of adjacent images in each group of images after the initial grouping, to determine the prevalence of the image; A secondary grouping unit, used for performing secondary grouping on the same group of images according to the prevalence to obtain a final image group; The image recognition module comprises: An extraction execution unit, configured to read the prevalence of each image in each image group, and extract at least one image according to the prevalence; A subject recognition unit is used to perform subject recognition on the extracted image to obtain a subject phrase; a sentence generation unit, for arranging the subject phrases of each image according to the image order of the image sequence, and converting the subject phrases into sentences based on a statistical language model; The sentence statistics unit is used to count all sentences and obtain evaluation texts; The extracting at least one image from each image group, identifying the image, and generating the evaluation text also includes: Pre-constructing a sample set of images to description sentences, and training a neural network model based on the sample set; When an image is recognized, the image and its corresponding sentence are used as new samples to update the sample set; Determine the error rate of the neural network model in real time according to the new sample, and adjust the application frequency of the fixed recognition scheme according to the error rate; Send new samples to the verification end regularly and receive sample correction instructions from the verification end.

5. The AI-based intelligent video analysis system according to claim 4 is characterized in that: The component extraction module comprises: A timeline creation unit, used to obtain the time span of the video to be identified and create a timeline; An audio information extraction unit, used for extracting audio information from an audio track of a video to be identified based on a time axis; An image insertion unit, used to obtain the frame rate of the video, select image points on the time axis based on the frame rate, read the image on the image track of the video to be identified, and establish a corresponding relationship with the image points; The image arrangement unit is used to count the corresponding images based on the sorting order of the image points to obtain an image sequence.

Images