Video quality detection method and device, equipment and storage medium

Abstract

The invention provides a video quality detection method, apparatus and device, and a storage medium. The method comprises the steps of obtaining a to-be-detected video for a target place; acquiring audio information in the to-be-detected video when it is determined that the to-be-detected video does not have repeated images, the to-be-detected video does not have image black edges, and the maximum continuous duration of the video in the to-be-detected video is greater than a preset duration; the first similarity between two adjacent frames of images in the maximum continuous duration of the video is greater than a first preset threshold value; converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one segmented word; for each segmented word, matching the segmented word with at least one keyword corresponding to the target place to obtain a matching result; and determining a quality detection result of the to-be-detected video based on each matching result. According to the invention, the video quality detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a video quality detection method, device, equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology and the popularization of smart mobile devices, short video platforms have become an important channel for people to share their lives and promote products and services. Among them, store exploration videos, as an emerging type of short video, provide consumers with an intuitive and vivid consumption guide through the personal experience and introduction of influencers.

[0003] Currently, before publishing these videos, the quality of the videos needs to be reviewed to ensure that their content is authentic, rich, and in compliance with platform specifications. The existing technology mainly relies on manual review. However, this review method is inefficient and difficult to cope with the rapid growth of massive video content. Summary of the invention

[0004] The present invention provides a video quality detection method, device, equipment and storage medium, which are used to solve the defect of low efficiency of video quality review in the prior art and improve the efficiency of video quality detection.

[0005] The present invention provides a video quality detection method, comprising: Obtain the video to be detected at the target location; Acquire audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; Converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one word segmentation; For each of the segmented words, matching the segmented word with at least one keyword corresponding to the target place to obtain a matching result; Based on each of the matching results, a quality detection result of the video to be detected is determined.

[0006] According to a video quality detection method provided by the present invention, determining whether the video to be detected does not have an image black edge includes: From all the images of the video to be detected, extract a frame of target image every preset number of frames; For any two adjacent frames of target images, determining a second similarity between the two adjacent frames of target images; Determine the latter frame of the target image of two adjacent frames of target images corresponding to the second similarity less than the second preset threshold as the key frame image; When there are no black edges in the key frame images greater than a first preset number, it is determined that there are no black edges in the video to be detected.

[0007] According to a video quality detection method provided by the present invention, determining that the video to be detected does not have repeated images includes: Traversing all key frame images, comparing the hash value of the currently traversed key frame image with the hash values ​​of all key frame images before the currently traversed key frame image, and obtaining a first comparison result; When all first comparison results indicate that there is no identical hash value, it is determined that there is no repeated image in the video to be detected.

[0008] According to a video quality detection method provided by the present invention, when it is determined that the video to be detected does not have repeated images, the video to be detected does not have black edges, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, obtaining audio information in the video to be detected includes: When it is determined that the video to be detected does not have repeated images, the video to be detected does not have black edges, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, the hash value of each key frame image in the video to be detected is compared with the hash value of the key frame image of each video in the video library to obtain a second comparison result; the video library stores videos obtained by video collection for other places of the same type as the target place; When the second comparison results of all videos indicate that the number of identical hash values ​​is not greater than a second preset number, audio information in the video to be detected is obtained.

[0009] According to a video quality detection method provided by the present invention, determining the quality detection result of the video to be detected based on each of the matching results includes: In the case that at least one matching result among all the matching results indicates that there is a target word segment matching the keyword, it is determined that the quality detection result is qualified in the quality detection.

[0010] According to a video quality detection method provided by the present invention, the method further includes: The video to be detected is published on a target platform, and the target segmentation is determined as a video tag of the video to be detected, and the video tag is used to search for the video to be detected on the target platform.

[0011] According to a video quality detection method provided by the present invention, the converting of the audio information into text information comprises: Inputting the audio information into an audio conversion model to obtain the text information output by the audio conversion model; Among them, the audio conversion model is obtained by optimizing the model parameters of the initial audio conversion model based on the audio segment corresponding to the erroneous text contained in the predicted text information and the correct text corresponding to the audio segment when the predicted text information obtained by inputting the audio sample collected for the target place into the initial audio conversion model is erroneous.

[0012] The present invention also provides a video quality detection device, comprising: An acquisition module is used to acquire the video to be detected for the target location; The acquisition module is further used to acquire audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; A conversion module, used for converting the audio information into text information; A processing module, used for performing word segmentation processing on the text information to obtain at least one word segmentation; A matching module, for matching each of the segmented words with at least one keyword corresponding to the target place to obtain a matching result; A determination module is used to determine the quality detection result of the video to be detected based on each of the matching results.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned video quality detection methods when executing the computer program.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the video quality detection method described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the video quality detection method described above is implemented.

[0016] The video quality detection method, device, equipment and storage medium provided by the present invention obtain the video to be detected for the target place, and when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration, obtain the audio information in the video to be detected, convert the audio information into text information, and perform word segmentation processing on the text information to obtain at least one word segmentation, and for each word segmentation, match the word segmentation with at least one keyword corresponding to the target place, and after obtaining the matching result, determine the quality detection result of the video to be detected based on each matching result. Since the video to be detected can be divided into image and audio information for detection respectively, and the audio information is further detected if the image is qualified, the quality of the video to be detected can be automatically detected based on the detection result of the image and the detection result of the audio information, avoiding the phenomenon of manually reviewing the video to be detected in the prior art, which can not only improve the accuracy of the quality detection of the video to be detected, but also improve the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a video quality detection method provided by an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of the structure of a video quality detection device provided by an embodiment of the present invention.

[0020] Figure 3 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] At present, before releasing the store exploration video, the video quality needs to be reviewed first, and the video will be released only when it passes the quality review. The current video quality review is mainly completed manually, however, manual review is inefficient and time-consuming, and the standards for manual review are different, which may lead to subjectivity and inconsistency in the review results, resulting in low accuracy of the review results.

[0023] In response to the above problems, an embodiment of the present invention provides a video quality detection method, in which, for a video to be detected collected at a target location, it is possible to determine whether the image in the video to be detected meets the quality requirements published by the platform based on the content of each frame of the image in the video to be detected. When it meets the quality requirements published by the platform, audio information is further separated from the video to be detected, so as to determine whether at least one keyword corresponding to the target location is mentioned in the audio file. When at least one keyword corresponding to the target location is mentioned, it indicates that the audio information also meets the quality requirements. By automatically detecting the quality of the video to be detected from both image and audio aspects, the phenomenon of manually reviewing video quality in the prior art is avoided, and the efficiency and accuracy of video quality review are improved.

[0024] Combine the following Figure 1 The video quality detection method provided by an embodiment of the present invention is described. The embodiment of the present invention can be applicable to video quality detection in any scenario, and can be particularly applied to the scenario of detecting the quality of store exploration videos. The executor of the method can be an electronic device such as an intelligent camera, a computer, a server, a server cluster, or a specially designed video quality detection device, or it can be a video quality detection device provided in the electronic device, and the video quality detection device can be implemented by software, hardware, or a combination of the two.

[0025] Figure 1 A schematic diagram of a flow chart of a video quality detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes: Step 101: Obtain a video to be detected for a target location.

[0026] In this step, the target place can be a shopping mall or a shop, or a tourist attraction. Video capture of the target place can obtain the video to be detected. In one scenario, after the video shooter shoots the video to be detected, the video to be detected can be saved. When the quality of the video to be detected needs to be reviewed, the opencv library of python is used to read the video to be detected through cv2.VideoCapture, and obtain the length and frames per second of the video to be detected. In addition, each frame of the video to be detected is stored in the image list, so that each frame of the video to be detected can be directly obtained from the image list later, so as to complete the quality detection of the video to be detected by detecting each frame of the image.

[0027] Step 102: When it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, obtain audio information in the video to be detected; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold.

[0028] In this step, in order to avoid the problem of multiple repeated images being played in a video to be detected and reducing the quality of the video to be detected, it is also necessary to detect whether there are repeated images in the video to be detected. Among them, repeated images can be understood as images with completely identical screen contents. When repeated images are detected in the video to be detected, it is determined that the quality of the video to be detected is unqualified, and a first prompt message will be output to remind the reviewer that there are repeated images in the video to be detected for rotation.

[0029] In addition, when capturing video of a target location, there may be black areas at the edges of the captured video screen due to improper shooting, unreasonable parameter settings, or inaccurate video cropping. The black areas are the black edges of the image.

[0030] In one implementation, each frame of the video to be detected can be judged for black edges using the Python opencv library. When there are more than a third preset number of images with black edges, it means that the quality of the video to be detected is unqualified, and a second prompt message will be output to remind the auditor that there are black edges in the images in the video to be detected. The third preset number can be set according to the actual situation or the length of the video to be detected. For example, the longer the video length, the larger the value of the third preset number.

[0031] In another implementation, key frame images can be determined from all images in the video to be detected, and the image black edges can be determined using the Python opencv library. When there are more than a fourth preset number of key frame images with image black edges, it means that the quality of the video to be detected is unqualified, and a second prompt message will be output to remind the auditor that there are image black edges in the images in the video to be detected. The fourth preset number can be set according to actual conditions or the number of key frame images. For example, the more key frame images there are, the larger the value of the fourth preset number.

[0032] Furthermore, for the video to be detected shot in the store exploration scenario, it is usually necessary to have continuous content exceeding a preset time in the video to be detected, so as to introduce a certain object in the store through the continuous content. If a video does not contain continuous content exceeding a preset time, it means that the introduction or shooting time of each object in the store in the video is short, and the video does not meet the specified quality requirements. Therefore, it is also necessary to determine the maximum continuous duration of the video based on the similarity between two adjacent frames of images in the video to be detected. The first similarity between any two adjacent frames of images within the maximum continuous duration of the video is greater than the first preset threshold, that is, the similarity of the images within the maximum continuous duration of the video is high. Among them, if the similarity between two adjacent frames of images is less than or equal to the first preset threshold, it means that the video is no longer continuous from the latter frame of the two adjacent frames of images, for example, the scene is changed or the shooting object is changed.

[0033] When the maximum continuous duration of the video is less than or equal to the preset duration, it means that the quality of the video to be tested is unqualified, and a third prompt message will be output to remind the reviewer that there is no continuous content exceeding the preset duration in the video to be tested.

[0034] The preset time length may be set according to actual conditions or experience. For example, when 30 frames of images are captured in one second, the preset time length may be set to 90 seconds.

[0035] When it is determined that there are no repeated images or black edges in the video to be detected, and the maximum continuous duration of the video is greater than the preset duration, it means that the image content in the video to be detected meets the quality requirements. At this time, the audio information in the video to be detected will be further detected. Among them, the audio information in the video to be detected can be understood as the commentary audio when shooting the target place. Exemplarily, by deploying ffmpeg, the audio information can be separated from the video to be detected using commands.

[0036] Step 103: convert the audio information into text information, and perform word segmentation on the text information to obtain at least one word segmentation.

[0037] In this step, after the audio information is separated, the audio information is converted into text information, for example, by a pre-trained audio conversion model or by any audio conversion application.

[0038] Furthermore, the obtained text information needs to be segmented to obtain at least one segmented word, wherein these segmented words can be keywords with high frequency in the text information. Exemplarily, the term frequency-inverse document frequency (TF-IDF) algorithm in the jieba package can be used to extract at least one segmented word in the text information.

[0039] Step 104: For each segmented word, match the segmented word with at least one keyword corresponding to the target place to obtain a matching result.

[0040] In this step, at least one keyword corresponding to each place may be pre-stored. For example, if the target place is a restaurant, at least one keyword may include the dishes of the restaurant. If the target place is a jewelry store, at least one keyword may include the name of the jewelry in the jewelry store, etc.

[0041] For each obtained word segment, the word segment is matched with at least one pre-stored keyword corresponding to the target place to obtain a matching result corresponding to each word segment.

[0042] Step 105: Based on each matching result, determine the quality detection result of the video to be detected.

[0043] In this step, after all the segmented words are matched, if there is a segmented word that matches successfully, it means that the audio information contains relevant content that mentions the target place, so that the audio information quality can be determined to be qualified, and thus the quality test result of the video to be tested can be determined to be qualified. When there is no segmented word that matches successfully, it means that the audio information does not mention relevant content of the target place, so that the audio information quality can be determined to be unqualified, and thus the quality test result of the video to be tested can be determined to be unqualified.

[0044] The video quality detection method provided by the embodiment of the present invention obtains the video to be detected for the target place, and when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration, obtains the audio information in the video to be detected, converts the audio information into text information, and performs word segmentation processing on the text information to obtain at least one word segmentation, and for each word segmentation, matches the word segmentation with at least one keyword corresponding to the target place, and after obtaining the matching result, determines the quality detection result of the video to be detected based on each matching result. Since the video to be detected can be divided into image and audio information for detection respectively, and the audio information is further detected if the image is qualified, the quality of the video to be detected can be automatically detected based on the detection result of the image and the detection result of the audio information, avoiding the phenomenon of manually reviewing the video to be detected in the prior art, which can not only improve the accuracy of the quality detection of the video to be detected, but also improve the detection efficiency.

[0045] Exemplarily, based on the above embodiment, when determining whether there are image black edges in the video to be detected, one frame of target image may be extracted from all the images in the video to be detected every preset number of frames, and for any two adjacent frames of target image, the second similarity of the two adjacent frames of target image is determined, and the latter frame of target image in the two adjacent frames of target image corresponding to the second similarity less than the second preset threshold is determined as a key frame image. When there are no image black edges in the key frame images greater than the first preset number, it is determined that there are no image black edges in the video to be detected.

[0046] Specifically, after the video to be detected is divided into multiple frames, in order to improve performance and detection efficiency, images with high similarity can be filtered, for example, by setting a preset number of frames, and extracting a frame of target image every preset number of frames from all images. The preset number of frames can be, for example, 10 frames.

[0047] In order to further reduce the amount of data detected, a key frame image can also be determined from the extracted target image. By determining the second similarity between any two adjacent target images, if the second similarity is less than a second preset threshold, it means that the similarity between the two target images is small, so that the latter of the two adjacent target images can be determined as a key frame image.

[0048] For example, based on the inter-frame difference method, the absolute value of the grayscale difference can be determined by subtracting the corresponding pixels of different frames. When the absolute value exceeds a certain threshold, it is considered to be a key frame. Specifically, the two adjacent target frames can be converted from red, green, and blue (RGB) images to grayscale images, and the absdiff of openCV can be called to obtain the absolute difference map of the current frame and the previous frame. The sum of the pixel values ​​in the difference map is calculated and divided by the shape of the difference map to obtain the average difference value. The average difference value is added as the Frame and the corresponding frame image to the window set to be processed. Among them, the average difference value is used instead of the difference map mainly because the average difference value has a small amount of data and high processing efficiency, which can simplify subsequent processing.

[0049] For the acquisition of key frame images, the difference threshold and local maxima principles are used respectively according to the playback length of the video to be detected. For the difference threshold, the average frame difference greater than the threshold is considered to be a key frame. By looping the window set to be processed, the relative change of the adjacent average difference values ​​is calculated. If the relative change is greater than the threshold, the frame image corresponding to the current average difference value is used as a key frame, otherwise the frame image is discarded. The above method is suitable for shorter videos, and the number of key frame images obtained can also be controlled by adjusting the threshold. In the local maxima method, the numpy Hanning algorithm is first used to eliminate the noise of the window set, and then the numpy argrelextrem function is used to obtain the vertices and set the window length. The window length is used to control the number of key frames to eliminate similar images. A window array only obtains the difference image of the maximum value (so it is called the local maximum). This method is suitable for longer videos, and the number of key frame images obtained can be controlled by adjusting the window length.

[0050] After the key frame images are determined, it is detected whether there are black edges in these key frame images. When it is determined that there are more than a first preset number of key frame images without black edges in all the key frame images, it can be determined that there are no black edges in the video to be detected. The first preset number can be determined according to the number of key frame images or can be set according to experience.

[0051] In this embodiment, by extracting a target image every preset number of frames from all the images of the video to be detected, the number of images for subsequent quality detection can be reduced, thereby improving the detection efficiency. In addition, key frame images can be determined from the extracted target images, and whether there are black edges in the key frame images can be detected. Since the key frame images are images with low similarity, relatively similar images can be filtered out, which can not only improve the efficiency of image quality detection, but also improve the accuracy of detection.

[0052] Exemplarily, based on the above embodiment, when determining whether there are repeated images in the video to be detected, all key frame images can be traversed, and the hash value of the currently traversed key frame image can be compared with the hash values ​​of all key frame images before the currently traversed key frame image to obtain a first comparison result. When all first comparison results indicate that there are no identical hash values, it is determined that there are no repeated images in the video to be detected.

[0053] Specifically, after determining the key frame images in the video to be detected, the hash value of each key frame image can be determined, and the determined hash value can be stored in a list. By traversing all the key frame images, the hash value of each traversed key frame image is compared with the hash values ​​of all key frame images before the currently traversed key frame image, and the first comparison result is obtained until the last key frame image is traversed. When all the first comparison results indicate that there is no identical hash value, it means that there is no identical image in the video to be detected, that is, there is no duplicate image.

[0054] It should be noted that, during the traversal process, once it is determined that the same hash value exists, the subsequent key frame images will not be traversed and compared further, and the quality of the video to be detected will be directly determined to be unqualified.

[0055] In this embodiment, the efficiency of quality detection can be improved by comparing only the hash values ​​of the key frame images. In addition, by calculating the hash values ​​of the key frame images and using the hash values ​​as a detection standard for whether the images are repeated, the accuracy of repeated image detection can be improved.

[0056] Exemplarily, on the basis of the above embodiments, when it is determined that the video to be detected does not have black edges on the image, does not have repeated images in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration, when obtaining the audio information in the video to be detected, it can be that when it is determined that the video to be detected does not have black edges on the image, does not have repeated images in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration, the hash value of each key frame image in the video to be detected is compared with the hash value of the key frame image of each video in the video library to obtain a second comparison result; the video library stores videos obtained by video capture for other places of the same type as the target place; when the second comparison results of all videos indicate that the number of the same hash value is not greater than the second preset number, the audio information in the video to be detected is obtained.

[0057] Specifically, in the above-mentioned embodiments, all belong to the mode of detecting the image of the video to be detected itself. In practical applications, there may be a phenomenon of shooting multiple videos to be detected for the same target place. Therefore, in order to eliminate the situation of duplicate videos, it is also necessary to compare the video to be detected with other videos in the video library. The video library stores the video fingerprint of the video obtained after video collection for other places of the same type as the target place, wherein the video fingerprint includes the address of the video, the hash value of the key frame image in the video, and the key frame image.

[0058] The hash value of each key frame image in the video to be detected is compared with the hash value of the key frame image of each video in the video library to obtain a second comparison result.

[0059] When the second comparison results of all videos in the video library indicate that the number of identical hash values ​​is not greater than the second preset number, it means that the similarity between the video to be detected and each video in the video library is small, that is, there is no video in the video library that is relatively similar to the video to be detected. In this case, it is determined that the image quality test of the video to be detected is qualified, and the audio information will be separated from the video to be detected for further detection of the audio information.

[0060] It is worth noting that when the image quality test of the video to be detected is qualified, the address of the video to be detected, the hash value of the key frame image of the video to be detected and the key frame image are used as video fingerprints and stored in the above video library.

[0061] In this embodiment, after quality detection of the image in the video to be detected, it is necessary to further compare the video to be detected with other videos in the video library, so as to filter out videos with a high degree of repetition with other videos, so as to make the quality detection result of the video to be detected more accurate.

[0062] Exemplarily, based on the above embodiments, when determining the quality inspection result of the video to be inspected based on each matching result, it may be that when all matching results include at least one matching result representing the existence of a target word matching the keyword, the quality inspection result is determined to be qualified.

[0063] Specifically, at least one word segment extracted from the text information after the audio information is converted is matched with at least one keyword corresponding to the target place. After obtaining the matching results, when it is determined based on the matching results that at least one target word segment and the keyword are successfully matched, it means that the audio information contains at least one keyword corresponding to the target place, and the video quality detection result of the video to be detected is determined to be qualified for the quality detection.

[0064] In this embodiment, by presetting at least one keyword corresponding to the target place and matching at least one word segment extracted from the text information corresponding to the audio information with the keyword corresponding to the target place, it is possible to quickly determine whether the target audio contains explanations related to the target place, thereby making the quality detection of the audio information more targeted and improving the results of the quality detection.

[0065] Exemplarily, based on the above embodiment, when the quality inspection result of the video to be inspected is qualified, the video to be inspected can also be published on the target platform, and the target segmentation can be determined as the video tag of the video to be inspected, and the video tag is used to search for the video to be inspected on the target platform.

[0066] Specifically, the target platform may include an application for publishing the video to be detected. When the video to be detected that has passed the quality inspection is released on the target platform, the target segmentation that successfully matches the keyword can be used as the video tag of the video to be detected. In this way, when other users search for videos on the target platform, they can quickly search for the video to be detected based on the video tag.

[0067] Exemplarily, based on the above embodiments, when converting audio information into text information, the audio information can be input into an audio conversion model to obtain text information output by the audio conversion model; wherein the audio conversion model is obtained by optimizing the model parameters of the initial audio conversion model based on the audio segment corresponding to the erroneous text contained in the predicted text information and the correct text corresponding to the audio segment when the predicted text information obtained by inputting the audio sample collected for the target place into the initial audio conversion model is erroneous.

[0068] Specifically, multiple sample audios can be collected in advance, and text labels can be annotated on the sample audios to obtain text labels. The sample audios can be input into a first audio conversion model to obtain predicted text output by the first audio conversion model. The predicted text and the annotated text labels can be compared to determine loss information, and the model parameters of the first audio conversion model can be adjusted based on the loss information. The above process is repeated continuously until the number of repetitions meets the preset number of times or the obtained model converges, and the final model is determined as the initial audio conversion model.

[0069] When converting audio information, the audio information can be input into the trained initial audio conversion model to obtain the corresponding text information. When the target location is a restaurant, since there may be no contextual information for the dish name or related introduction, this particularity will cause errors in text conversion. When the audio sample collected at the target location is input into the above-mentioned trained initial audio conversion model, if the predicted text information obtained is erroneous, the audio segment corresponding to the erroneous text can be intercepted from the audio sample, and the correct text corresponding to the audio segment can be determined by manual annotation. Furthermore, the audio segment can be input into the initial audio conversion model to obtain new predicted text information, and new loss information can be determined based on the new predicted text information and the correct text, so that the model parameters of the initial audio conversion model can be fine-tuned according to the new loss information, that is, the model parameters of the initial audio conversion model are optimized to obtain the audio conversion model.

[0070] The audio conversion model obtained in the above manner can re-optimize the model parameters based on the audio segments with conversion errors. Therefore, after the audio information is converted into text based on the audio conversion model, the obtained text information has higher accuracy.

[0071] The video quality detection device provided by the present invention is described below. The video quality detection device described below and the video quality detection method described above can be referred to each other.

[0072] Figure 2 A schematic diagram of the structure of a video quality detection device provided by an embodiment of the present invention, referring to Figure 2 As shown, the video quality detection device 200 includes: An acquisition module 11 is used to acquire a video to be detected for a target location; The acquisition module 11 is further used to acquire audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; A conversion module 12, used for converting the audio information into text information; A processing module 13 is used to perform word segmentation processing on the text information to obtain at least one word segmentation; A matching module 14 is used to match each of the segmented words with at least one keyword corresponding to the target place to obtain a matching result; The determination module 15 is used to determine the quality detection result of the video to be detected based on each of the matching results.

[0073] In an exemplary embodiment, the determination module 15 is further configured to: From all the images of the video to be detected, extract a frame of target image every preset number of frames; For any two adjacent frames of target images, determining a second similarity between the two adjacent frames of target images; Determine the latter frame of the target image of two adjacent frames of target images corresponding to the second similarity less than the second preset threshold as the key frame image; When there are no black edges in the key frame images greater than a first preset number, it is determined that there are no black edges in the video to be detected.

[0074] In an exemplary embodiment, the determination module 15 is further configured to: Traversing all key frame images, comparing the hash value of the currently traversed key frame image with the hash values ​​of all key frame images before the currently traversed key frame image, and obtaining a first comparison result; When all first comparison results indicate that there is no identical hash value, it is determined that there is no repeated image in the video to be detected.

[0075] In an exemplary embodiment, the acquisition module 11 is specifically configured to: When it is determined that the video to be detected does not have repeated images, the video to be detected does not have black edges, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, the hash value of each key frame image in the video to be detected is compared with the hash value of the key frame image of each video in the video library to obtain a second comparison result; the video library stores videos obtained by video collection for other places of the same type as the target place; When the second comparison results of all videos indicate that the number of identical hash values ​​is not greater than a second preset number, audio information in the video to be detected is obtained.

[0076] In an exemplary embodiment, the determination module 15 is specifically configured to: In the case that at least one matching result among all the matching results indicates that there is a target word segment matching the keyword, it is determined that the quality detection result is qualified in the quality detection.

[0077] In an exemplary embodiment, the apparatus further comprises: a publishing module, wherein: A publishing module, used for publishing the video to be detected on a target platform; The determination module 15 is further used to determine the target segmented word as a video tag of the video to be detected, and the video tag is used to search for the video to be detected on the target platform.

[0078] In an exemplary embodiment, the conversion module 12 is specifically configured to: Inputting the audio information into an audio conversion model to obtain the text information output by the audio conversion model; Among them, the audio conversion model is obtained by optimizing the model parameters of the initial audio conversion model based on the audio segment corresponding to the erroneous text contained in the predicted text information and the correct text corresponding to the audio segment when the predicted text information obtained by inputting the audio sample collected for the target place into the initial audio conversion model is erroneous.

[0079] The device of this embodiment can be used to execute the method of any embodiment in the video quality detection method side embodiment. Its specific implementation process and technical effects are similar to those in the video quality detection method side embodiment. For details, please refer to the detailed description in the video quality detection method side embodiment, which will not be repeated here.

[0080] Figure 3 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the video quality detection method, which includes: obtaining a video to be detected for a target place; obtaining audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration; the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one word segmentation; for each word segmentation, matching the word segmentation with at least one keyword corresponding to the target place to obtain a matching result; and determining the quality detection result of the video to be detected based on each matching result.

[0081] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video quality detection method provided by the above-mentioned methods, which includes: obtaining a video to be detected for a target place; when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, obtaining audio information in the video to be detected; the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one word segmentation; for each word segmentation, matching the word segmentation with at least one keyword corresponding to the target place to obtain a matching result; and determining the quality detection result of the video to be detected based on each matching result.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the video quality detection method provided by the above-mentioned methods, the method comprising: obtaining a video to be detected for a target place; obtaining audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration; the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one word segmentation; for each word segmentation, matching the word segmentation with at least one keyword corresponding to the target place to obtain a matching result; and determining the quality detection result of the video to be detected based on each matching result.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video quality detection method, characterized in that: include: Obtain the video to be detected at the target location; Acquire audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; Converting the audio information into text information, and performing word segmentation processing on the text information to obtain at least one word segmentation; For each of the segmented words, matching the segmented word with at least one keyword corresponding to the target place to obtain a matching result; Based on each of the matching results, a quality detection result of the video to be detected is determined.

2. The video quality detection method according to claim 1, characterized in that: Determining that the video to be detected does not have a black edge on the image includes: From all the images of the video to be detected, extract a frame of target image every preset number of frames; For any two adjacent frames of target images, determining a second similarity between the two adjacent frames of target images; Determine the latter frame of the target image of two adjacent frames of target images corresponding to the second similarity less than the second preset threshold as the key frame image; When there are no black edges in the key frame images greater than a first preset number, it is determined that there are no black edges in the video to be detected.

3. The video quality detection method according to claim 2, characterized in that: The determining that the video to be detected does not have a repeated image includes: Traversing all key frame images, comparing the hash value of the currently traversed key frame image with the hash values ​​of all key frame images before the currently traversed key frame image, and obtaining a first comparison result; When all first comparison results indicate that there is no identical hash value, it is determined that there is no repeated image in the video to be detected.

4. The video quality detection method according to claim 1, characterized in that: The method of obtaining audio information in the video to be detected, when it is determined that the video to be detected does not have repeated images, the video to be detected does not have black edges, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, includes: When it is determined that the video to be detected does not have repeated images, the video to be detected does not have black edges, and the maximum continuous duration of the video in the video to be detected is greater than a preset duration, the hash value of each key frame image in the video to be detected is compared with the hash value of the key frame image of each video in the video library to obtain a second comparison result; the video library stores videos obtained by video collection for other places of the same type as the target place; When the second comparison results of all videos indicate that the number of identical hash values ​​is not greater than a second preset number, audio information in the video to be detected is obtained.

5. The video quality detection method according to claim 1, characterized in that: The step of determining the quality detection result of the video to be detected based on each of the matching results includes: In the case that at least one matching result among all the matching results indicates that there is a target word segment matching the keyword, it is determined that the quality detection result is qualified in the quality detection.

6. The video quality detection method according to claim 5, characterized in that: The method further comprises: The video to be detected is published on a target platform, and the target segmentation is determined as a video tag of the video to be detected, and the video tag is used to search for the video to be detected on the target platform.

7. The video quality detection method according to any one of claims 1 to 6, characterized in that: The converting the audio information into text information comprises: Inputting the audio information into an audio conversion model to obtain the text information output by the audio conversion model; Among them, the audio conversion model is obtained by optimizing the model parameters of the initial audio conversion model based on the audio segment corresponding to the erroneous text contained in the predicted text information and the correct text corresponding to the audio segment when the predicted text information obtained by inputting the audio sample collected for the target place into the initial audio conversion model is erroneous.

8. A video quality detection device, characterized in that: include: An acquisition module is used to acquire the video to be detected for the target location; The acquisition module is further used to acquire audio information in the video to be detected when it is determined that there are no repeated images in the video to be detected, there are no black edges in the video to be detected, and the maximum continuous duration of the video in the video to be detected is greater than the preset duration; and the first similarity between two adjacent frames of images within the maximum continuous duration of the video is greater than a first preset threshold; A conversion module, used for converting the audio information into text information; A processing module, used for performing word segmentation processing on the text information to obtain at least one word segmentation; A matching module, for matching each of the segmented words with at least one keyword corresponding to the target place to obtain a matching result; A determination module is used to determine the quality detection result of the video to be detected based on each of the matching results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the video quality detection method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video quality detection method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video quality detection method according to any one of claims 1 to 7 is implemented.

Images