Model self-adaption test method and device, electronic equipment and storage medium

Abstract

This invention discloses a model adaptive testing device, electronic device, and storage medium. The method includes: acquiring multiple video test samples; inputting a first set of video frames and a second set of video frames from the video test samples into a video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first set of video frames and a second model prediction result corresponding to the second set of video frames; determining pseudo-classification labels corresponding to the video test samples based on audio information in the video test samples; determining a target loss value based on the first model prediction result, the second model prediction result, and the pseudo-classification labels; and correcting the model parameters in the video classification model based on the target loss value to obtain a tested video classification model. This technical solution achieves the effect of updating and adjusting the video classification model by introducing audio information present in the video, thereby improving the model performance and robustness of the adjusted video classification model.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a model adaptive testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, the field of machine learning has made significant progress. For many machine learning problems, training deep neural networks on large amounts of data has yielded very powerful deep models with excellent performance. Examples include facial recognition, image recognition, and autonomous driving.

[0003] Currently, deep models can be applied not only to the image domain but also to video-based tasks to process video data.

[0004] However, in practical applications, videos are easily affected by irrelevant factors (such as weather, lighting, and background changes), which can lead to distribution shifts in test data. Furthermore, video may lose its temporal action information after being interfered with by noise. When using deep models to process videos, it is difficult to adapt to their variability, resulting in low recognition rates and poor robustness. Summary of the Invention

[0005] This invention provides a model adaptive testing method, apparatus, electronic device, and storage medium to update and adjust a video classification model by incorporating audio information present in the video, thereby improving the performance and robustness of the adjusted video classification model.

[0006] According to one aspect of the present invention, a model adaptive testing method is provided, the method comprising:

[0007] Multiple video test samples are obtained, wherein the video test samples include a first video frame set, a second video frame set, and audio information corresponding to the sample video. The first video frame set and the second video frame set each include the same number of video frames. The first video frame set corresponds to a first sampling interval, and the second video frame set corresponds to a second sampling interval. The first sampling interval and the second sampling interval are not equal.

[0008] For each video test sample, the first video frame set and the second video frame set in the current video test sample are respectively input into the video classification model to be tested, to obtain the first model prediction result corresponding to the first video frame set and the second model prediction result corresponding to the second video frame set; and

[0009] Based on the audio information in the current video test sample, determine the pseudo-classification label corresponding to the current video test sample;

[0010] Based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until the preset test target is reached, thus obtaining a video classification model that has completed the test.

[0011] According to another aspect of the present invention, a model adaptive testing apparatus is provided, the apparatus comprising:

[0012] The sample acquisition module is used to acquire multiple video test samples, wherein the video test samples include a first video frame set, a second video frame set, and audio information corresponding to the sample video. The first video frame set and the second video frame set each include the same number of video frames. The first video frame set corresponds to a first sampling interval, and the second video frame set corresponds to a second sampling interval. The first sampling interval and the second sampling interval are not equal.

[0013] The prediction result determination module is used to, for each video test sample, input the first video frame set and the second video frame set in the current video test sample into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set; and...

[0014] The label determination module is used to determine the pseudo-classification label corresponding to the current video test sample based on the audio information in the current video test sample;

[0015] The model generation module is used to determine the target loss value based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label, and to correct the model parameters in the video classification model based on the target loss value until the preset test target is reached, thereby obtaining a video classification model that has been tested.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the model adaptive testing method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the model adaptive testing method according to any embodiment of the present invention.

[0021] The technical solution of this invention involves acquiring multiple video test samples. Further, for each video test sample, the first and second video frame sets in the current video test sample are input into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. Based on the audio information in the current video test sample, a pseudo-classification label corresponding to the current video test sample is determined. Then, based on the first model prediction result, the second model prediction result, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until a preset test target is reached, resulting in a completed video classification model. This solves the problem in related technologies where video classification models are easily affected by video distribution migration in practical applications, leading to low accuracy in video category classification. It achieves the effect of introducing audio information present in the video to update and adjust the video classification model, thereby improving the model performance and robustness of the adjusted video classification model. This improves the adaptability of the video classification model to the actual application environment, thus achieving the effect of improving video classification accuracy and efficiency.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a model adaptive testing method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of a model adaptive testing method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a flowchart of a model adaptive testing method provided in Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a model adaptive testing device according to Embodiment 4 of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the model adaptive testing method of this invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that the technical solutions provided in the embodiments of this invention can be applied to the model testing process. Simply put, a pre-trained video classification model can be retrieved, and video test samples can be constructed. Then, the pre-trained video classification model can be tested based on the video test samples, and the model parameters in the pre-trained video classification model can be corrected. This results in a tested video classification model. To ensure that the final video classification model can adapt to changes in the distribution of the videos to be classified and reduce the impact of distribution shift on video classification, the video test samples can be improved, and the pre-trained video classification model can be tested based on the improved video test samples. Finally, the tested video classification model can be used as the final desired video classification model.

[0032] Example 1

[0033] Figure 1This is a flowchart of a model adaptive testing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where model parameters are adjusted during model testing to adapt the adjusted model to the actual application environment. This method can be executed by a model adaptive testing device, which can be implemented in hardware and / or software and can be configured in a terminal and / or server. Figure 1 As shown, the method includes:

[0034] S110. Obtain multiple video test samples, wherein the video test samples include a first set of video frames, a second set of video frames, and audio information corresponding to the sample videos.

[0035] In this embodiment, the video test sample can be understood as the sample used in the neural network model testing process. The sample video can be any video. Optionally, the sample video can be a video captured in real time by a camera device, or a video reconstructed by a video reconstruction model, or a video pre-stored in the storage space. The first video frame set can be understood as the set of video frames obtained after sampling the sample video. The second video frame set can be understood as the set of video frames obtained after sampling the sample video. Both the first and second video frame sets include the same number of video frames. The first and second video frame sets correspond to different sampling intervals, that is, the first video frame set corresponds to the first sampling interval, and the second video frame set corresponds to the second sampling interval. The first sampling interval and the second sampling interval are not equal. The audio information can be understood as the information obtained after extracting audio from the sample video.

[0036] In this embodiment, before testing the model, multiple video test samples can be obtained to test the model based on these samples. To improve the model's accuracy and robustness, as many and varied video test samples as possible can be obtained. These video test samples can be constructed based on sample videos; therefore, multiple video test samples can be constructed first.

[0037] Based on this, in addition to the above technical solutions, the method further includes: acquiring multiple sample videos; for each sample video, sampling the current sample video according to a first sampling interval and a preset sampling number to obtain a first video frame set; sampling the current sample video according to a second sampling interval and a preset sampling number to obtain a second video frame set; extracting audio from the current sample video to obtain sample audio, and sampling the sample audio according to a preset sampling frequency to obtain audio information; and constructing a video test sample corresponding to the current sample video based on the first video frame set, the second video frame set, and the audio information.

[0038] It should be noted that when processing video resources for video classification, including all video resources in the classification process would consume a significant amount of computing resources. Furthermore, because videos change continuously, there is information redundancy between frames; video frames with closer time intervals contain more similar information, so it is unnecessary to retain all of them. Sampling and extracting frames from the video, such as equal frame extraction or keyframe extraction, can effectively reduce the amount of video image data to be processed while preserving complete video information, thus improving data processing efficiency.

[0039] In this embodiment, the first sampling interval can be any sampling interval. Optionally, the first sampling interval can be 5 frames / time. The preset sampling number can be any number. Optionally, the preset sampling number can be 10 frames. The second sampling interval can be any sampling interval. Optionally, the second sampling interval can be 10 frames / time, etc. The preset sampling frequency can be a pre-set frequency for sampling audio. The preset sampling frequency can be any sampling frequency. Optionally, the preset sampling frequency can be 16 kHz, etc.

[0040] As an optional implementation in this invention, multiple sample videos can be acquired. Further, for each sample video, the current sample video can be sampled according to a first sampling interval to extract a preset number of video frames from the current sample video, and the multiple video frames are arranged in timestamp order. This yields a first video frame set. Also, the current sample video can be sampled according to a second sampling interval to extract a preset number of video frames from the current sample video, and the extracted multiple video frames are arranged in timestamp order. This yields a second video frame set. Furthermore, audio can be extracted from the current sample video based on a preset audio extraction method, and the extracted audio can be used as sample audio. Furthermore, the sample audio can be sampled according to a preset sampling frequency, and the sampled information is used as audio information corresponding to the current sample video. Further, a sample set can be constructed based on the first video frame set, the second video frame set, and the audio information, and the constructed sample set is used as the video test sample corresponding to the current sample video.

[0041] Furthermore, multiple video test samples can be constructed based on the above method, and the video classification model to be tested can be tested based on multiple video test samples.

[0042] S120. For each video test sample, input the first video frame set and the second video frame set in the current video test sample into the video classification model to be tested, respectively, to obtain the first model prediction result corresponding to the first video frame set and the second model prediction result corresponding to the second video frame set.

[0043] It should be noted that each video test sample can be tested using the S120 method. This will result in a completed video classification model.

[0044] In this embodiment, the video classification model to be tested can be a neural network model capable of video classification that has completed the model training phase and is to be tested. The video classification model can be understood as a neural network model that takes video information as input and classifies videos based on actions within the video information. The video classification model can be any model structure and can achieve video classification. The first model prediction result can be understood as the video frame processing result output after inputting a first set of video frames into the video classification model to be tested. The first model prediction result may include a first predicted category and a first feature statistic. The first predicted category can be a sample video category prediction result determined based on the first set of video frames. The first feature statistic may include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model. Optionally, the feature parameters may include at least one of variance and mean. The second model prediction result can be understood as the video frame processing result output after inputting a second set of video frames into the video classification model to be tested. The second model prediction result may include a second predicted category and a second feature statistic. The second predicted category can be a sample video category prediction result determined based on the second set of video frames. The second feature statistic may include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model.

[0045] As an optional implementation in this invention, for each video test sample, the first video frame set and the second video frame set in the current video test sample can be input into the video classification model to be tested, respectively. Then, the first video frame set and the second video frame set can be processed based on the video classification model, and a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set can be output.

[0046] S130. Based on the audio information in the current video test sample, determine the pseudo-classification label corresponding to the current video test sample.

[0047] It should be noted that S120 and S130 do not have a temporal execution relationship; these two steps can be executed in parallel or sequentially. When these two steps are executed sequentially, step S120 can be executed first, followed by step S130; or step S130 can be executed first, followed by step S120. This embodiment does not specifically limit this.

[0048] In this embodiment, the pseudo-classification label can be understood as the video category prediction result that is closest to the true label.

[0049] It's important to note that in practical applications, video recording often involves simultaneous audio and video, with a close connection between the two. Furthermore, audio is less susceptible to noise from lighting conditions, camera shake, or other sources of visual disturbance. Since audio is typically emitted by the subject moving within the video, audio information can be used to assist the video classification model in adapting to test video samples to complete the video action recognition task. It's also worth noting that during model testing, the test video samples do not include the actual video classification labels. Therefore, pseudo-classification labels can be determined based on audio information to correct the model parameters of the video classification model under test. The advantages of determining pseudo-classification labels based on audio information for video test samples include: improved accuracy of the pseudo-classification labels, increased similarity between pseudo-classification labels and actual labels, and consequently, improved model performance and robustness of the video classification model.

[0050] As an optional implementation in this embodiment, audio information from the current video test sample can be obtained. Then, audio classification processing can be performed on the audio information to determine at least one audio tag corresponding to the audio information and a probability value corresponding to each audio tag. Further, based on the at least one audio tag and the probability value corresponding to each audio tag, a video classification tag closest to the audio information can be determined in a pre-constructed video classification tag space. Then, the determined video classification tag can be used as a pseudo-classification tag corresponding to the current video test sample.

[0051] S140. Based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification labels, determine the target loss value, and correct the model parameters in the video classification model based on the target loss value until the preset test target is reached, thus obtaining the tested video classification model.

[0052] In this embodiment, the target loss value can be understood as a numerical value characterizing the degree of difference between the prediction results of the first model, the prediction results of the second model, and the pseudo-classification labels. The preset test target can be a pre-set goal to be achieved in the model testing phase. Optionally, the preset test target can be the convergence of the loss function in the video classification model; or it can be reaching a preset number of iterations, etc.

[0053] In this embodiment, after obtaining the prediction results of the first model, the prediction results of the second model, and the pseudo-classification labels, the target loss value can be determined based on these results. Furthermore, the model parameters in the video classification model can be corrected based on the target loss value. Further, if the preset test target is achieved, the model test is considered complete, and the tested video classification model is used as the final applied video classification model.

[0054] In this embodiment, the target loss value can be a loss value determined based on multiple loss functions. The process of determining the target loss value will be explained below.

[0055] Optionally, based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label, a target loss value is determined, including: performing loss processing on the first feature statistic or the second feature statistic and the pre-acquired training feature statistic according to the first loss function to obtain a first loss value; performing loss processing on the first predicted category and the second predicted category according to the second loss function to obtain a second loss value; performing loss processing on the first predicted category or the second predicted category and the pseudo-classification label according to the third loss function to obtain a third loss value; and determining the target loss value based on the first loss value, the second loss value, and the third loss value.

[0056] In this embodiment, the first loss function can be any loss function. Optionally, the first loss function can be an L1 norm loss function. The second loss function can be any loss function. Optionally, the second loss function can be a consistency loss function. The third loss function can be any loss function. Optionally, the third loss function can be a classification loss function, such as a cross-entropy loss function.

[0057] In this embodiment, training feature statistics can be understood as the feature parameter statistics obtained during the model training phase. Training feature statistics can include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model. The parameter values ​​of the feature parameters corresponding to each layer are the average values ​​of the feature parameters determined based on multiple training samples. That is, during the model training phase, for each training sample, the current training sample can be input into the video classification model to be trained. Then, feature statistics corresponding to the current training sample can be output, which can include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model. Further, after obtaining the feature statistics corresponding to each training sample, the feature statistics can be summed to obtain the total feature statistics, which can include the total parameter values ​​of the feature parameters corresponding to each layer in the video classification model. Then, the total feature statistics can be averaged based on the number of training samples, and the final determined average can be used as the training feature statistics.

[0058] As an optional implementation in this embodiment, in order to make the feature statistics obtained in the testing phase closer to those obtained in the training phase, the feature statistics obtained in these two phases can be aligned. Then, the model parameters in the video classification model to be tested can be corrected based on the feature alignment result. It should be noted that the first feature statistic corresponds to the first video frame set, and the second feature statistic corresponds to the second video frame set. Both the first and second video frame sets are included in the video test samples. Therefore, when performing feature alignment, the feature statistics corresponding to any video frame set in the video test samples can be processed with the feature statistics obtained in the training phase to obtain the loss value.

[0059] In practical applications, the training feature statistics obtained during the training phase can be obtained in advance. After obtaining the first feature statistics and the second feature statistics, one feature statistics can be arbitrarily selected from the first feature statistics and the second feature statistics. The first loss function is then used to process the loss of the feature statistics and the training feature statistics to obtain the first loss value.

[0060] For example, the first loss function can be represented based on the following formula:

[0061]

[0062] in, represents the first loss function; l represents the number of layers in the video classification network; The parameter value representing the mean of the l-th layer in the first or second feature statistic; The parameter value representing the mean value corresponding to the l-th layer in the training feature statistics; The parameter value represents the variance corresponding to the l-th layer in the first or second feature statistic; The parameter value represents the variance corresponding to the l-th layer in the training feature statistics.

[0063] In this embodiment, to ensure that the prediction results obtained by the video classification model for the same sample video using different data augmentation methods are as consistent as possible, a consistency loss processing can be applied to the first and second predicted categories. Furthermore, the model parameters in the video classification model can be adjusted based on the loss processing results.

[0064] As an optional implementation method in this embodiment, after obtaining the first predicted category and the second predicted category, a second loss function can be used to process the loss of the first predicted category and the second predicted category to obtain a second loss value.

[0065] In this embodiment, to determine the degree of difference between the pseudo-classification label determined based on audio information and the first or second predicted category, loss processing can be applied to the pseudo-classification label and the first predicted category, or loss processing can be applied to the pseudo-classification label and the second predicted category. Furthermore, the model parameters in the video classification model can be corrected based on the loss processing results.

[0066] As an optional implementation method in this embodiment, after obtaining the first predicted category, the second predicted category, and the pseudo-classification label, a third loss function can be used to process the loss of the first predicted category and the pseudo-classification label to obtain a third loss value; or, a third loss function can be used to process the loss of the second predicted category and the pseudo-classification label to obtain a third loss value.

[0067] In this embodiment, after obtaining the first loss value, the second loss value, and the third loss value, the target loss value can be determined based on the first loss value, the second loss value, and the third loss value.

[0068] Optionally, based on the first loss value, the second loss value, and the third loss value, a target loss value is determined, including: determining a first weight value corresponding to the first loss value, determining a second weight value corresponding to the second loss value, and determining a third weight value corresponding to the third loss value; and performing a weighted summation on the first loss value, the first weight value, the second loss value, the second weight value, the third loss value, and the third weight value to obtain the target loss value.

[0069] In this embodiment, the first weight value can be understood as the weight value corresponding to the first loss value determined based on the first loss function. The second weight value can be understood as the weight value corresponding to the second loss value determined based on the second loss function. The third weight value can be understood as the weight value corresponding to the third loss value determined based on the third loss function.

[0070] As an optional implementation in this embodiment, a first weight value corresponding to the first loss value, a second weight value corresponding to the second loss value, and a third weight value corresponding to the third loss value can be determined. Further, the first loss value and the first weight value can be multiplied to obtain a first value to be superimposed; the second loss value and the second weight value can be multiplied to obtain a second value to be superimposed; and the third loss value and the third weight value can be multiplied to obtain a third value to be superimposed. Further, the first value to be superimposed, the second value to be superimposed, and the third value to be superimposed can be added together to obtain the target loss value.

[0071] For example, the target loss value can be determined based on the following formula:

[0072] L=αL align +βL cons+γL cls

[0073] Where L represents the target loss value; α represents the first weight value; L align This represents the first loss function used to determine the first loss value; β represents the second weight value; L cons This represents the second loss function used to determine the second loss value; γ represents the third weight value; L cls This represents the third loss function used to determine the third loss value.

[0074] Furthermore, when refining the model parameters in the video classification model, the convergence of the loss function can be used as a test objective. This could include checking if the test error is less than a preset error, if the error trend stabilizes, or if the current number of iterations equals a preset number. If convergence is achieved—for example, if the test error of the loss function is less than a preset error, or if the error trend stabilizes—it indicates that the video classification model test is complete, and iterative testing can be stopped. If convergence has not yet been achieved, other video test samples can be obtained to test the video classification model until the loss function converges. When the loss function in the video classification model converges, the tested video classification model can be used as the final required video classification model. That is, inputting the video to be classified into the video classification model will accurately yield the corresponding video recognition category.

[0075] The technical solution of this invention involves acquiring multiple video test samples. Further, for each video test sample, the first and second video frame sets in the current video test sample are input into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. Based on the audio information in the current video test sample, a pseudo-classification label corresponding to the current video test sample is determined. Then, based on the first model prediction result, the second model prediction result, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until a preset test target is reached, resulting in a completed video classification model. This solves the problem in related technologies where video classification models are easily affected by video distribution migration in practical applications, leading to low accuracy in video category classification. It achieves the effect of introducing audio information present in the video to update and adjust the video classification model, thereby improving the model performance and robustness of the adjusted video classification model. This improves the adaptability of the video classification model to the actual application environment, thus achieving the effect of improving video classification accuracy and efficiency.

[0076] Example 2

[0077] Figure 2This is a flowchart of a model adaptive testing method provided in Embodiment 2 of the present invention. Based on the foregoing embodiments, at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag can be determined. Further, based on at least one audio tag, the probability values ​​corresponding to each audio tag, and a pre-constructed video tag space, a pseudo-classification tag corresponding to the video test sample is determined. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or similar to those in the above embodiments will not be repeated here.

[0078] like Figure 2 As shown, the method includes:

[0079] S210. Obtain multiple video test samples, wherein the video test samples include a first set of video frames, a second set of video frames, and audio information corresponding to the sample videos.

[0080] S220. For each video test sample, input the first video frame set and the second video frame set in the current video test sample into the video classification model to be tested, respectively, to obtain the first model prediction result corresponding to the first video frame set and the second model prediction result corresponding to the second video frame set.

[0081] S230. Determine at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag.

[0082] In this embodiment, the audio information in the current video test sample can be processed using a preset audio classification method. This allows for the acquisition of at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag. The preset audio classification method can be any audio classification method. Optionally, the preset audio classification method can be based on an audio classification model, etc.

[0083] Optionally, determining at least one audio label corresponding to the audio information and the probability value corresponding to each audio label includes: retrieving a pre-trained audio classification model; and inputting the audio information into the audio classification model to determine at least one audio label corresponding to the audio information and the probability value corresponding to each audio label.

[0084] In this embodiment, the audio classification model can be a neural network model that takes audio information as input to identify the audio information and determine its classification label. Specifically, the audio classification model is trained on a neural network model based on sample audio and the corresponding real labels.

[0085] In practical applications, a pre-trained audio classification model can be directly retrieved. Furthermore, the audio information can be input into the retrieved audio classification model for processing. This allows the generation of at least one audio label corresponding to the audio information and the probability values ​​for each label.

[0086] S240. Based on at least one audio label, the probability value corresponding to each audio label, and the pre-constructed video label space, determine the pseudo-classification label corresponding to the current video test sample.

[0087] The video tag space can be a set of at least one video classification tag. The video classification tags can be predetermined tags used to characterize video categories.

[0088] In practical applications, after obtaining at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag, the at least one audio tag, the probability value corresponding to each audio tag, and the pre-constructed video tag space can be processed to determine the video classification tag that is closest to the audio information from the video tag space based on the audio tag and the probability value corresponding to each audio tag, and the video classification tag is used as the pseudo classification tag corresponding to the current video test sample.

[0089] Optionally, based on at least one audio tag, the probability value corresponding to each audio tag, and a pre-determined video tag space, a pseudo-classification tag corresponding to the current video test sample is determined, including: processing at least one audio tag, the probability value corresponding to each audio tag, and the video tag space according to a preset prompt text template to obtain a target prompt text; and inputting the target prompt text into a language processing model to obtain a pseudo-classification tag corresponding to the current video test sample.

[0090] In this embodiment, the preset prompt text template can be a pre-set template used to input into the language processing model so that the model can output the required results. Optionally, the preset prompt text template can include at least five editable items. The at least five editable items can include a background introduction editable item, a task description editable item, an example editable item, a task requirement editable item, and an input editable item. Among them, the background introduction editable item and the task description editable item are used to describe the specific details of the task to be performed by the model. The example editable item is used to describe specific examples so that the model can better understand the task to be performed. The task requirement editable item is used to emphasize the key content of the task and standardize the format of the model output. The input editable item can include at least one audio tag corresponding to the obtained audio information, the probability value corresponding to each audio tag, and the video tag space. It should be noted that the content included in the background introduction editable item, the task description editable item, the example editable item, and the task requirement editable item can all be described using natural language. In this embodiment, the target prompt text can be the text obtained after all the editable items in the preset prompt text template have been entered. For example, the target prompt text can be "<Background Introduction><Task Description><Example><Task Requirements><Input: Audio Tag" and probability value Video category tag space Y v >。

[0091] The language processing model can be a deep learning model trained on text data. This model can perform intent recognition on the input natural language, process the natural language based on the intent recognition results, and then output the text information required by the user. In this embodiment, the language processing model can be a deep learning model that determines the video classification label that best matches the audio information based on the input prompt text.

[0092] In practical applications, after obtaining at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag, the corresponding text content can be edited for each editable item included in the preset prompt text template. The edited text can then be used as the target prompt text. Furthermore, the target prompt text can be input into a language processing model to process it and determine at least one video classification tag in the video tag space. This yields at least one video classification tag, which can be used as a pseudo-classification tag corresponding to the current video test sample.

[0093] S250. Based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification labels, determine the target loss value, and correct the model parameters in the video classification model based on the target loss value until the preset test target is reached, thus obtaining the tested video classification model.

[0094] The technical solution of this invention involves acquiring multiple video test samples. Further, for each video test sample, the first and second video frame sets in the current video test sample are input into the video classification model to be tested, respectively. This yields a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. Then, at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag are determined. Based on the at least one audio tag, the probability values ​​corresponding to each audio tag, and a pre-constructed video tag space, a pseudo-classification tag corresponding to the current video test sample is determined. Then, based on the first model prediction result, the second model prediction result, and the pseudo-classification tag, a target loss value is determined. The model parameters in the video classification model are then corrected based on the target loss value until a preset test target is reached, resulting in a tested video classification model. This achieves the effect of updating and adjusting the video classification model by introducing audio information present in the video, thereby improving the model performance and robustness of the adjusted video classification model. This, in turn, improves the adaptability of the video classification model to the actual application environment, thus achieving the effect of improving video classification accuracy and efficiency.

[0095] Example 3

[0096] Figure 3 This is a flowchart illustrating a model adaptive testing method provided in Embodiment 3 of the present invention. Based on the foregoing embodiments, after obtaining the tested video classification model, a video to be classified can be acquired, and the video to be classified can be processed based on the video classification model to obtain the video recognition category corresponding to the video to be classified. Specific implementation methods can be found in the technical solution of this embodiment. Technical terms that are the same as or similar to those in the above embodiments will not be repeated here.

[0097] like Figure 3 As shown, the method includes:

[0098] S310. Obtain multiple video test samples, wherein the video test samples include a first set of video frames, a second set of video frames, and audio information corresponding to the sample videos.

[0099] S320. For each video test sample, input the first video frame set and the second video frame set in the current video test sample into the video classification model to be tested, respectively, to obtain the first model prediction result corresponding to the first video frame set and the second model prediction result corresponding to the second video frame set.

[0100] S330. Based on the audio information in the current video test sample, determine the pseudo-classification label corresponding to the current video test sample.

[0101] S340. Based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification labels, determine the target loss value, and correct the model parameters in the video classification model based on the target loss value until the preset test target is reached, thus obtaining the tested video classification model.

[0102] S350, Obtain the video to be categorized.

[0103] In this embodiment, the video to be classified can be any video to be classified. Optionally, the video to be classified can be a video received by the server or client and captured in real time by a pre-set camera device; or it can be a stored video retrieved by the server or client from a relevant database, etc.

[0104] In practical applications, videos to be classified can be obtained. Then, video classification can be performed on the videos based on the neural network model provided in this embodiment of the invention.

[0105] S360. Based on the third sampling interval, the video to be classified is sampled to obtain a set of video frames to be classified.

[0106] In this embodiment, the third sampling interval can be any one of the first and second sampling intervals, or it can be a different sampling interval than the first and second sampling intervals. The set of video frames to be classified can be a set including multiple video frames.

[0107] In practical applications, after obtaining the video to be classified, the video to be classified can be sampled according to the third sampling interval to extract multiple video frames to be classified from the video to be classified. The extracted multiple video frames to be classified are then arranged in the order of timestamps to obtain the set of video frames to be classified.

[0108] S370. Input the set of video frames to be classified into the video classification model after the test is completed to obtain the video recognition category corresponding to the video to be classified.

[0109] In this embodiment, the video recognition category can be understood as the category to which the video belongs, determined based on the model's identification of a set of video frames. For example, the video recognition category may include playing basketball, playing soccer, playing badminton, or swimming, etc.

[0110] In practical applications, after obtaining the set of video frames to be classified, this set can be input into a tested video classification model to process the set of video frames. This allows the determination of the video recognition category corresponding to the video to be classified.

[0111] The technical solution of this invention involves acquiring multiple video test samples. Further, for each video test sample, the first and second video frame sets in the current video test sample are respectively input into the video classification model to be tested, obtaining a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. Then, based on the audio information in the current video test sample, a pseudo-classification label corresponding to the current video test sample is determined. Subsequently, based on the first model prediction result, the second model prediction result, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until a preset test target is reached, resulting in a tested video classification model. Further, a video to be classified is acquired, and then the video to be classified is sampled according to a third sampling interval to obtain a set of video frames to be classified. This set of video frames is then input into the tested video classification model to obtain the video recognition category corresponding to the video to be classified. This reduces the impact of distribution offset in the video on the video classification model and improves the video classification accuracy.

[0112] Example 4

[0113] Figure 4 This is a schematic diagram of the structure of a model adaptive testing device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device includes: a sample acquisition module 410, a prediction result determination module 420, a pseudo-label determination module 430, and a model generation module 440.

[0114] The system includes a sample acquisition module 410 for acquiring multiple video test samples, each video test sample comprising a first video frame set, a second video frame set, and audio information corresponding to a sample video. Both the first and second video frame sets contain the same number of video frames. The first video frame set corresponds to a first sampling interval, and the second video frame set corresponds to a second sampling interval. The first and second sampling intervals are not equal. A prediction result determination module 420, for each video test sample, inputs the first and second video frame sets from the current video test sample into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. A pseudo-label determination module 430, based on the audio information in the current video test sample, determines a pseudo-classification label corresponding to the current video test sample. A model generation module 440, based on the first model prediction result, the second model prediction result, and the pseudo-classification label, determines a target loss value and corrects the model parameters in the video classification model based on the target loss value until a preset test target is reached, thus obtaining a completed video classification model.

[0115] The technical solution of this invention involves acquiring multiple video test samples. Further, for each video test sample, the first and second video frame sets in the current video test sample are input into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set. Based on the audio information in the current video test sample, a pseudo-classification label corresponding to the current video test sample is determined. Then, based on the first model prediction result, the second model prediction result, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until a preset test target is reached, resulting in a completed video classification model. This solves the problem in related technologies where video classification models are easily affected by video distribution migration in practical applications, leading to low accuracy in video category classification. It achieves the effect of introducing audio information present in the video to update and adjust the video classification model, thereby improving the model performance and robustness of the adjusted video classification model. This improves the adaptability of the video classification model to the actual application environment, thus achieving the effect of improving video classification accuracy and efficiency.

[0116] Optionally, the device further includes: a sample video acquisition module, a first video sampling module, a second video sampling module, an audio extraction module, and a test sample construction module.

[0117] The sample video acquisition module is used to acquire multiple sample videos;

[0118] The first video sampling module is used to sample the current sample video according to the first sampling interval and the preset sampling number for each sample video to obtain the first video frame set;

[0119] The second video sampling module samples the current sample video according to the second sampling interval and the preset sampling number to obtain a second video frame set;

[0120] The audio extraction module is used to extract audio from the current sample video to obtain sample audio, and to sample the sample audio according to a preset sampling frequency to obtain audio information;

[0121] The test sample construction module is used to construct a video test sample corresponding to the current sample video based on the first video frame set, the second video frame set, and the audio information.

[0122] Optionally, the pseudo-label determination module 430 includes: an audio label determination unit and a pseudo-classification label determination unit.

[0123] An audio tag determination unit is used to determine at least one audio tag corresponding to the audio information and a probability value corresponding to each audio tag;

[0124] The pseudo-classification label determination unit is used to determine the pseudo-classification label corresponding to the current video test sample based on the at least one audio label, the probability value corresponding to each audio label, and a pre-constructed video label space, wherein the video label space includes at least one video classification label.

[0125] Optionally, the audio tag determination unit includes: a model retrieval subunit and an audio tag determination subunit.

[0126] The model retrieval subunit is used to retrieve a pre-trained audio classification model;

[0127] An audio tag determination subunit is used to input the audio information into the audio classification model to determine at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag.

[0128] Optionally, the pseudo-classification label determination unit includes: a text prompt determination subunit and a pseudo-classification label determination subunit.

[0129] The text prompt determination subunit is used to process the at least one audio tag, the probability value corresponding to each audio tag, and the video tag space according to the preset text prompt template to obtain the target text prompt;

[0130] The pseudo-classification label determination subunit is used to input the target text prompt into the language processing model to obtain pseudo-classification labels corresponding to the current video test sample.

[0131] Optionally, the first model prediction result includes a first prediction category and a first feature statistic, and the second model prediction result includes a second prediction category and a second feature statistic; both the first feature statistic and the second feature statistic include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model, and the feature parameters include at least one of variance and mean;

[0132] The model generation module 440 includes: a first loss value determination unit, a second loss value determination unit, a third loss value determination unit, and a target loss value determination unit.

[0133] The first loss value determination unit is used to perform loss processing on the first feature statistic or the second feature statistic and the pre-acquired training feature statistic according to the first loss function to obtain the first loss value.

[0134] The second loss value determination unit is used to perform loss processing on the first prediction category and the second prediction category according to the second loss function to obtain the second loss value;

[0135] The third loss value determination unit is used to perform loss processing on the first predicted category or the second predicted category and the pseudo-classification label according to the third loss function to obtain the third loss value.

[0136] The target loss value determination unit is used to determine the target loss value based on the first loss value, the second loss value, and the third loss value.

[0137] Optionally, the target loss value determination unit includes: a weight value determination subunit and a target loss value determination subunit.

[0138] The weight value determination subunit is used to determine a first weight value corresponding to the first loss value, a second weight value corresponding to the second loss value, and a third weight value corresponding to the third loss value;

[0139] The target loss value determination subunit is used to perform a weighted summation of the first loss value, the first weight value, the second loss value, the second weight value, the third loss value, and the third weight value to obtain the target loss value.

[0140] Optionally, the device further includes: a video acquisition module, a video sampling module, and a recognition category determination module.

[0141] The video acquisition module is used to acquire videos to be categorized.

[0142] The video sampling module is used to sample the video to be classified according to the third sampling interval to obtain a set of video frames to be classified.

[0143] The identification category determination module is used to input the set of video frames to be classified into the video classification model that has been tested and completed, so as to obtain the video identification category corresponding to the video to be classified.

[0144] The model adaptive testing device provided in this embodiment of the invention can execute the model adaptive testing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0145] Example 5

[0146] Figure 5A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0147] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0148] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0149] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as model adaptive testing methods.

[0150] In some embodiments, the model adaptation testing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the model adaptation testing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the model adaptation testing method by any other suitable means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0155] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0156] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0157] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0158] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A model adaptive testing method, characterized in that, include: Multiple video test samples are obtained, wherein the video test samples include a first video frame set, a second video frame set, and audio information corresponding to the sample video. The first video frame set and the second video frame set each include the same number of video frames. The first video frame set corresponds to a first sampling interval, and the second video frame set corresponds to a second sampling interval. The first sampling interval and the second sampling interval are not equal. For each video test sample, the first video frame set and the second video frame set in the current video test sample are respectively input into the video classification model to be tested, to obtain the first model prediction result corresponding to the first video frame set and the second model prediction result corresponding to the second video frame set; and Based on the audio information in the current video test sample, determine the pseudo-classification label corresponding to the current video test sample; Based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label, a target loss value is determined, and the model parameters in the video classification model are corrected based on the target loss value until the preset test target is reached, thus obtaining a video classification model that has completed the test.

2. The method according to claim 1, characterized in that, Also includes: Acquire multiple sample videos; For each sample video, the current sample video is sampled according to the first sampling interval and the preset sampling number to obtain the first set of video frames; The current sample video is sampled according to the second sampling interval and the preset sampling number to obtain a second video frame set; Audio is extracted from the current sample video to obtain sample audio, and the sample audio is sampled according to a preset sampling frequency to obtain audio information; Based on the first set of video frames, the second set of video frames, and the audio information, a video test sample corresponding to the current sample video is constructed.

3. The method according to claim 1, characterized in that, The step of determining the pseudo-classification label corresponding to the current video test sample based on the audio information in the current video test sample includes: Determine at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag; Based on the at least one audio tag, the probability value corresponding to each audio tag, and the pre-constructed video tag space, a pseudo-classification tag corresponding to the current video test sample is determined, wherein the video tag space includes at least one video classification tag.

4. The method according to claim 3, characterized in that, Determining at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag includes: Retrieve the pre-trained audio classification model; The audio information is input into the audio classification model to determine at least one audio tag corresponding to the audio information and the probability value corresponding to each audio tag.

5. The method according to claim 3, characterized in that, The step of determining the pseudo-classification label corresponding to the current video test sample based on the at least one audio label, the probability value corresponding to each audio label, and a pre-determined video label space includes: The target text prompt is obtained by processing the at least one audio tag, the probability value corresponding to each audio tag, and the video tag space according to the preset text prompt template; The target text prompt is input into the language processing model to obtain pseudo-classification labels corresponding to the current video test sample.

6. The method according to claim 1, characterized in that, The first model prediction result includes a first prediction category and a first feature statistic, and the second model prediction result includes a second prediction category and a second feature statistic; both the first feature statistic and the second feature statistic include the parameter values ​​of the feature parameters corresponding to each layer in the video classification model, and the feature parameters include at least one of variance and mean; The step of determining the target loss value based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label includes: The first loss value is obtained by applying a loss function to the first feature statistic or the second feature statistic and the pre-acquired training feature statistic. The first predicted category and the second predicted category are processed according to the second loss function to obtain the second loss value; The first predicted category or the second predicted category and the pseudo-classification label are subjected to loss processing based on the third loss function to obtain the third loss value; The target loss value is determined based on the first loss value, the second loss value, and the third loss value.

7. The method according to claim 6, characterized in that, Determining the target loss value based on the first loss value, the second loss value, and the third loss value includes: A first weight value corresponding to the first loss value is determined, a second weight value corresponding to the second loss value is determined, and a third weight value corresponding to the third loss value is determined; The first loss value, the first weight value, the second loss value, the second weight value, the third loss value, and the third weight value are weighted and summed to obtain the target loss value.

8. The method according to claim 1, characterized in that, Also includes: Get the videos to be categorized; The video to be classified is sampled according to the third sampling interval to obtain a set of video frames to be classified. The set of video frames to be classified is input into the video classification model that has been tested, and the video recognition category corresponding to the video to be classified is obtained.

9. A model adaptive testing device, characterized in that, include: The sample acquisition module is used to acquire multiple video test samples, wherein the video test samples include a first video frame set, a second video frame set, and audio information corresponding to the sample video. The first video frame set and the second video frame set each include the same number of video frames. The first video frame set corresponds to a first sampling interval, and the second video frame set corresponds to a second sampling interval. The first sampling interval and the second sampling interval are not equal. The prediction result determination module is used to, for each video test sample, input the first video frame set and the second video frame set in the current video test sample into the video classification model to be tested, respectively, to obtain a first model prediction result corresponding to the first video frame set and a second model prediction result corresponding to the second video frame set; and... The label determination module is used to determine the pseudo-classification label corresponding to the current video test sample based on the audio information in the current video test sample; The model generation module is used to determine the target loss value based on the prediction results of the first model, the prediction results of the second model, and the pseudo-classification label, and to correct the model parameters in the video classification model based on the target loss value until the preset test target is reached, thereby obtaining a video classification model that has been tested.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the model adaptive testing method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the model adaptive testing method according to any one of claims 1-8.

Images