Video processing method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a video processing method and device, electronic equipment and computer readable storage medium, and relate to the technical field of multimedia. The method comprises: identifying at least one key frame image from the video data being played; wherein the video data is generated based on original video encryption; the key frame image is a frame image with an encryption mark in the video data; for each key frame image, the encryption mark is decrypted to obtain decryption information corresponding to the encryption mark; when the decryption information meets a preset playing condition, the video data is continued to be played; otherwise, the video data is stopped to be played. The embodiments of the present application encrypt and monitor the original video through the key frame image, i.e. a single frame, effectively improve the completeness and smoothness of the video display picture, and improve the user experience.

Description

Technical Field

[0001] This application relates to the field of multimedia technology, and more specifically, to a video processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Advances in multimedia technology have driven the rapid development of the media industry. Media platforms such as radio, television, and the internet have an increasingly significant impact on people's daily lives, making the monitoring of video data from these platforms crucial. For example, monitoring the broadcast of television programs and identifying missed or delayed broadcasts can facilitate later verification. Alternatively, monitoring advertisements, which are typically played on a loop, requires advertisers to determine whether the ad playback meets their intended delivery requirements.

[0003] Taking advertising monitoring as an example, when monitoring advertising videos in public areas, existing technologies generally overlay a preset marker image directly onto the advertising video screen and detect the advertising video based on the marker image. However, in order to improve detection accuracy, the preset marker image usually covers the video screen over a large area for a long time, affecting the integrity and smoothness of the video screen. Summary of the Invention

[0004] This application provides a video processing method, apparatus, electronic device, and computer-readable storage medium, which can solve the problem of low video integrity and smoothness during video monitoring in the prior art. The technical solution is as follows:

[0005] According to one aspect of the embodiments of this application, a video processing method is provided, the method comprising:

[0006] Identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; and the keyframe image is a frame image in the video data that contains encryption markers.

[0007] For each keyframe image, the encrypted marker is decrypted to obtain the decrypted information corresponding to the encrypted marker;

[0008] If the decrypted information meets the preset playback conditions, continue playing the video data; otherwise, stop playing the video data.

[0009] In one possible implementation, the aforementioned video data is generated by encryption in the following manner:

[0010] Extract raw frame images from the original video;

[0011] Add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image;

[0012] The original frame images are replaced with keyframe images to generate video data.

[0013] In one possible implementation, the aforementioned video data is generated by encryption in the following manner:

[0014] Obtain the preset encrypted image;

[0015] Based on a preset time interval and timestamp, encrypted images are inserted as keyframe images into the original video to obtain video data.

[0016] In another possible implementation, the identification of at least one keyframe image includes:

[0017] The outer border region of each frame of the video data is detected, and when an encrypted marker is identified, the frame image is determined as a keyframe image.

[0018] In another possible implementation, the identification of at least one keyframe image includes:

[0019] Based on time intervals and timestamps, frame images are extracted from video data as keyframe images.

[0020] In yet another possible implementation, the identification of at least one keyframe image includes:

[0021] Based on a pre-trained recognition model, each frame of the video data is identified to obtain the category of the frame image; whereby the category is used to characterize whether there is an encrypted tag in the frame image.

[0022] If the category indicates that a cryptographic marker is present in the frame image, then the frame image is used as the keyframe image;

[0023] The recognition model was trained in the following way:

[0024] Obtain a preset sample image and its label category; the label category is used to characterize whether there is an encrypted marker in the sample image.

[0025] Input the sample image into the preset initial model to obtain the sample category in real time;

[0026] The parameters of the initial model are updated based on a preset loss function until the difference between the sample class and the label class meets the preset convergence condition, thus obtaining the recognition model.

[0027] In yet another possible implementation, the aforementioned preset playback conditions include at least one of the following:

[0028] Within the preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with the preset target decryption information.

[0029] Count the number of decrypted information that matches the target decrypted information, and ensure that the proportion of this number in the total number of keyframe images in the video data is not less than a preset threshold.

[0030] According to another aspect of the embodiments of this application, a video processing apparatus is provided, the apparatus comprising:

[0031] The recognition module is used to identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; and the keyframe image is a frame image in the video data that contains encryption markers.

[0032] The decryption module is used to decrypt the encrypted markers for each keyframe image to obtain the decrypted information corresponding to the encrypted markers;

[0033] The playback control model is used to continue playing video data when the decrypted information meets the preset playback conditions; otherwise, it stops playing video data.

[0034] In one possible implementation, the aforementioned video data is generated by encryption in the following manner:

[0035] Extract raw frame images from the original video;

[0036] Add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image;

[0037] The original frame images are replaced with keyframe images to generate video data.

[0038] In one possible implementation, the aforementioned video data is generated by encryption in the following manner:

[0039] Obtain the preset encrypted image;

[0040] Based on a preset time interval and timestamp, encrypted images are inserted as keyframe images into the original video to obtain video data.

[0041] In another possible implementation, when the above recognition module recognizes at least one keyframe image, it is used to:

[0042] The outer border region of each frame of the video data is detected, and when an encrypted marker is identified, the frame image is determined as a keyframe image.

[0043] In another possible implementation, when the above recognition module recognizes at least one keyframe image, it is used to:

[0044] Based on time intervals and timestamps, frame images are extracted from video data as keyframe images.

[0045] In yet another possible implementation, when the above-mentioned recognition module recognizes at least one keyframe image, it is used to:

[0046] Based on a pre-trained recognition model, each frame of the video data is identified to obtain the category of the frame image; whereby the category is used to characterize whether there is an encrypted tag in the frame image.

[0047] If the category indicates that a cryptographic marker is present in the frame image, then the frame image is used as the keyframe image;

[0048] The recognition model was trained in the following way:

[0049] Obtain a preset sample image and its label category; the label category is used to characterize whether there is an encrypted marker in the sample image.

[0050] Input the sample image into the preset initial model to obtain the sample category in real time;

[0051] The parameters of the initial model are updated based on a preset loss function until the difference between the sample class and the label class meets the preset convergence condition, thus obtaining the recognition model.

[0052] In yet another possible implementation, the aforementioned preset playback conditions include at least one of the following:

[0053] Within the preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with the preset target decryption information.

[0054] Count the number of decrypted information that matches the target decrypted information, and ensure that the proportion of this number in the total number of keyframe images in the video data is not less than a preset threshold.

[0055] According to another aspect of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method shown in the first aspect of the present application.

[0056] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method shown in the first aspect of the embodiments of this application.

[0057] According to one aspect of the present application, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps of the method shown in the first aspect of the present application.

[0058] The beneficial effects of the technical solutions provided in this application are:

[0059] This application embodiment identifies at least one keyframe image from the playing video data and decrypts the encryption marker of each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, the video data continues to play; otherwise, playback stops. The video data is generated based on the encryption of the original video, and the keyframe image is a frame image in the video data containing the encryption marker. This application embodiment achieves encryption and monitoring of the original video based on keyframe images, i.e., single frames. This differs from existing technologies that directly overlay marker images over a large area and for a long time in the original video frame for video monitoring. This application obtains decryption information by decrypting keyframe images and then determines whether to continue playing the video data based on the decryption information. Since the display frequency of a single frame in the video data is much lower than the refresh rate of the human eye, keyframe images are not easily perceived by the human eye, effectively improving the integrity and smoothness of the video display and enhancing the user experience. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0061] Figure 1 This is a schematic diagram illustrating an application scenario of a video processing method provided in an embodiment of this application;

[0062] Figure 2 A flowchart illustrating a video processing method provided in an embodiment of this application;

[0063] Figure 3 A schematic diagram illustrating the process of encrypting original video in a video processing method provided in this application embodiment;

[0064] Figure 4 A schematic diagram illustrating the process of identifying keyframe images in a video processing method provided in this application embodiment;

[0065] Figure 5 A flowchart illustrating the training of a recognition model in a video processing method provided in this application embodiment;

[0066] Figure 6 A flowchart illustrating an example video processing method provided in an embodiment of this application;

[0067] Figure 7 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of this application;

[0068] Figure 8 This is a schematic diagram of the structure of a video processing electronic device provided in an embodiment of this application. Detailed Implementation

[0069] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0070] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0072] With the development of media broadcasting and television, people's demand for multimedia video monitoring is also increasing. For example, media companies or program production companies use systems to automatically identify and register video broadcasts of television programs, directly generating television broadcast monitoring reports and analysis charts; record individual or all product programs for easy reference, analysis, and selection of products and time slots; and advertising companies monitor the playback of advertisements.

[0073] When monitoring broadcast television, manual intervention is usually required to monitor and statistically process video data, which often results in low monitoring efficiency and inflexible monitoring methods. When monitoring advertisements in public places, the system can cover the advertisement video screen with the logo image for a long time and a large area to achieve automatic monitoring of the advertisement, but this can affect the display effect of the advertisement.

[0074] The video processing method, apparatus, electronic device, and computer-readable storage medium provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0075] This application provides a video processing method, which can be implemented by a terminal or a server. The terminal or server involved in this application uses a high-speed camera to identify at least one keyframe image from the playing video data, and decrypts the encrypted marker of each keyframe image to obtain decrypted information. When the decrypted information does not meet preset playback conditions, the terminal or server issues an alarm to stop playing the video data.

[0076] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0077] like Figure 1 As shown, the video processing method of this application can be applied to... Figure 1 In the scenario shown, specifically, terminal 102 encrypts the original video to generate video data and plays it. Terminal 102 takes a screenshot of the playing video data and sends the screenshot to server 101, enabling server 101 to identify at least one keyframe image based on the video screenshot. The keyframe image is a frame image in the video data containing an encrypted marker. Server 101 decrypts the encrypted marker for each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, server 101 controls the terminal to continue playing the video data based on playback control instructions; otherwise, it controls terminal 102 to stop playing the video data.

[0078] Figure 1 In the scenario shown, the above video processing method can be performed on the server; in other scenarios, it can also be performed on the terminal.

[0079] Those skilled in the art will understand that the “terminal” used herein can be a mobile phone, tablet computer, PDA (Personal Digital Assistant), MID (Mobile Internet Device), etc.; and the “server” can be implemented using a standalone server or a server cluster composed of multiple servers.

[0080] This application provides a video processing method, such as... Figure 2 As shown, this method, which can be applied to servers or terminals performing video processing, includes:

[0081] S201, Identify at least one keyframe image from the video data being played.

[0082] Among them, the video data is generated based on the encryption of the original video; the keyframe image is a frame image in the video data that contains encryption markers.

[0083] Specifically, a server or terminal used for video processing can acquire a set of target frame images from the playing video data through a high-speed camera or screenshot control, identify the set of target frame images, and select at least one key frame image from it.

[0084] A high-speed camera is a device capable of capturing moving images at an exposure time of less than 1 / 1000 of a second or a frame rate exceeding 250 frames per second. It is used to record fast-moving objects as photographic images onto a storage medium. The high-speed camera in this embodiment can capture images of the display screen showing playing video data at a preset rate to obtain target frame images.

[0085] Furthermore, the video data can be played on a preset display screen, which can be independent of the server or terminal performing video processing, or integrated into the server or terminal performing video processing; this is not specifically limited in this embodiment. When the display screen is independent of the server or terminal performing video processing, the server or terminal performing video processing can establish a communication connection with the display screen based on a wireless local area network (WLAN) or a wired local area network (LAN); wherein, the wired LAN can be Ethernet based on the IEEE 802.3 (a LAN communication standard) protocol, and the WLAN can be Wi-Fi (a wireless communication technology) based on the IEEE 802.11 (a wireless network communication standard) protocol.

[0086] S202, for each keyframe image, the encrypted marker is decrypted to obtain the decryption information corresponding to the encrypted marker.

[0087] The aforementioned encryption marker may include at least one of the following: a QR code, a barcode, or an encoded string.

[0088] Specifically, for each keyframe image, the server or terminal used for video processing can extract the corresponding encrypted marker, and then decrypt the encrypted marker to obtain decrypted information; each keyframe image corresponds to one decrypted information.

[0089] In some implementations, the encryption mark is a QR code or barcode, and the server or terminal used for video processing can decrypt the encryption mark based on barcode recognition.

[0090] Barcode recognition technology refers to the technology of identifying barcodes using photoelectric conversion equipment. A barcode is a sequence of wide bars, narrow bars, and spaces, which can represent specific numerical and alphanumeric codes. A QR code, also known as a two-dimensional barcode, can store more information and represent more data types than a traditional barcode. Servers or terminals used for video processing can scan and read these barcodes or QR codes, perform photoelectric conversion, and then decode them based on the corresponding encoding rules to obtain the decrypted information.

[0091] In other embodiments, the aforementioned encryption token is an encoded string; the server or terminal used for video processing can pre-establish a correspondence between the string and the decryption information, and map the encoded string to the decryption information based on the aforementioned correspondence.

[0092] In this embodiment, the video data can be either an advertisement video or a television program video. Taking an advertisement video as an example, its decryption information may include the advertisement name, release time, release path, playback location, playback device number, etc., which are not specifically limited in this embodiment.

[0093] S203: If the decrypted information meets the preset playback conditions, continue playing the video data; otherwise, stop playing the video data.

[0094] Specifically, the server or terminal used for video processing can compare the decryption information with preset target decryption information. If the decryption information matches the target decryption information, the decryption result of the keyframe image is determined to be correct; if the decryption information does not match the target decryption information, or if the decryption information indicates that decryption is not possible, the decryption result of the keyframe image is incorrect; if the decryption result of at least one keyframe image matches the preset playback conditions, the video data continues to be played; otherwise, the video data playback stops.

[0095] In this embodiment, playback conditions can be set based on the decryption results corresponding to keyframe images within a preset playback time period, or based on the decryption results corresponding to all keyframe images in the video data. No specific limitations are imposed in this embodiment.

[0096] Furthermore, taking the monitoring of an advertising video as an example, it can be set that if the decryption result of three consecutive keyframe images is an error, the advertising video will stop playing; or it can be set that if the probability of the decryption result being correct is 90% or higher for all keyframe images in the advertising video, the advertising video will continue playing.

[0097] This application embodiment identifies at least one keyframe image from the playing video data and decrypts the encryption marker of each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, the video data continues to play; otherwise, playback stops. The video data is generated based on the encryption of the original video, and the keyframe image is a frame image in the video data containing the encryption marker. This application embodiment achieves encryption and monitoring of the original video based on keyframe images, i.e., single frames. This differs from existing technologies that directly overlay marker images over a large area and for a long time in the original video frame for video monitoring. This application obtains decryption information by decrypting keyframe images and then determines whether to continue playing the video data based on the decryption information. Since the display frequency of a single frame in the video data is much lower than the refresh rate of the human eye, keyframe images are not easily perceived by the human eye, effectively improving the integrity and smoothness of the video display and enhancing the user experience.

[0098] This application provides one possible implementation method, such as... Figure 3 As shown, the above video data was generated using the following encryption method:

[0099] S301, extract the original frame image from the original video.

[0100] Specifically, a server or terminal used for video processing can extract original frame images from the original video based on preset extraction rules, or it can randomly extract original frame images from the original video.

[0101] In this embodiment, taking an advertising video as an example, the total duration of the advertising video can be 120 seconds, with 60 frames per second. The server or terminal used for video processing can extract the 20th frame of the current second when the advertising video has played for a multiple of 10 seconds, and use this as the extracted original frame image. That is, 12 original frame images can be extracted from the advertising video.

[0102] S302, add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image.

[0103] The aforementioned outer border area can be the area corresponding to the BM (Black Matrix) region of the liquid crystal display screen. This outer border area does not overlap with the original frame image and is within the display area of ​​the screen.

[0104] Specifically, for liquid crystal display panels, the colors come from red, green, and blue color filters. In order to control each individual display unit (such as an R, i.e., a red filter, called a sub-pixel), there are control signal traces between sub-pixels and in the edge areas. In order to prevent the traces from affecting the display effect, the area other than the pixel (the three RGB sub-pixels are collectively called a pixel) is made into a black matrix area, i.e., the BM area. The function of the BM area can include preventing light leakage of the liquid crystal display and wiring for internal data signals and scanning signals of the liquid crystal display.

[0105] In this embodiment of the application, when the encryption mark is a barcode or a QR code, the encryption mark can be added to the outer border area, i.e., the BM area, of the original frame image based on different grayscale values ​​to generate a keyframe image.

[0106] S303 replaces the corresponding original frame image with the keyframe image to generate video data.

[0107] In this embodiment, encryption of the original frame image is achieved by adding an encryption mark to the outer border area of ​​the original frame image. Since the outer border area does not overlap with the original frame image and is within the display range of the display screen, the encryption process in this embodiment does not affect the display of the original video, thus ensuring the smoothness and integrity of the video.

[0108] This application provides a possible implementation method in which the above video data is encrypted and generated based on the following method:

[0109] Obtain a preset encrypted image; based on a preset time interval and timestamp, insert the encrypted image as a keyframe image into the original video to obtain video data.

[0110] The encrypted image includes encryption tags.

[0111] Specifically, a server or terminal used for video processing can locate the insertion position of a keyframe image based on a preset time interval and timestamp, and then insert the encrypted image as a keyframe image into the original video to obtain video data.

[0112] In this embodiment, the encrypted image is inserted as a single frame into the original video. Since the display frequency of a single frame in the video data is much lower than the refresh frequency of the human eye, the key frame image is not easily perceived by the human eye, thus ensuring the smoothness of the video and improving the user experience.

[0113] This application provides a possible implementation method in which at least one keyframe image is identified, including:

[0114] The outer border region of each frame of the video data is detected, and when an encrypted marker is identified, the frame image is determined as a keyframe image.

[0115] Specifically, the server or terminal used for video processing can capture the video data being played using a high-speed camera, acquiring each frame of the video data. Then, the outer border area of ​​each frame is detected; when an encryption mark is identified in the outer border area, the frame is determined as a keyframe image.

[0116] Furthermore, the aforementioned encryption mark may include a barcode or QR code. The server or terminal used for video processing can perform photoelectric conversion based on the grayscale value of the outer border area of ​​each frame image to identify the encryption mark.

[0117] This application embodiment detects the outer border area of ​​the frame image to identify the key frame image. Since the outer border area does not overlap with the original frame image and is within the display range of the screen, the smoothness and integrity of video playback can be guaranteed. At the same time, the identification of key frames based on the fixed area of ​​the frame image, namely the outer border area, effectively improves the efficiency of video processing.

[0118] This application provides a possible implementation method in which at least one keyframe image is identified, including:

[0119] Based on time intervals and timestamps, frame images are extracted from video data as keyframe images.

[0120] Specifically, the server or terminal used for video processing can determine the location information of keyframe images based on the aforementioned time interval and timestamp, and extract keyframe images from the video data according to the aforementioned location information.

[0121] In this embodiment of the application, taking a total video data duration of 120s as an example, when the time interval is 20s, the start timestamp is 20s, and the end timestamp is 100s, the location information for locating the key frame image includes: 20s, 40s, 60s, 80s, and 100s. The key frame image can then be determined at the time point determined by the above location information.

[0122] This application embodiment can directly extract keyframe images from video data by using preset time intervals and timestamps, which improves the accuracy of keyframe image recognition and lays a good foundation for the accuracy of subsequent video monitoring.

[0123] This application provides one possible implementation method, such as... Figure 4 As shown, the above-mentioned identification of at least one keyframe image includes:

[0124] S401, based on a pre-trained recognition model, identifies each frame of video data to obtain the category of the frame image.

[0125] The category is used to characterize whether a cryptographic marker exists in the frame image. The above recognition model can be a model based on machine learning algorithms.

[0126] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, and complex algorithms. It uses computers as tools and aims to realistically and in real-time simulate human learning methods, while also structuring existing content to effectively improve learning efficiency.

[0127] Taking a convolutional neural network model as an example, it can include an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The input layer is used to extract image features from each frame; the convolutional layer is used to scale the image features; the pooling layer can be used to select features and filter information from the scaled image features; the fully connected layer is used to integrate the feature information output by the pooling layer to reduce data redundancy; the output layer can perform feature classification based on a preset classification function to obtain the category of the frame image.

[0128] S402, when the category indicates that there is an encryption marker in the frame image, the frame image is used as the keyframe image.

[0129] Among them, such as Figure 5 As shown, the recognition model is trained in the following way:

[0130] S501, Obtain the preset sample image and the label category of the sample image.

[0131] The label category is used to characterize whether there are encrypted tags in the sample image.

[0132] S502, input the sample image into the preset initial model to obtain the sample category in real time;

[0133] S503, update the parameters of the initial model based on the preset loss function until the difference between the sample category and the label category meets the preset convergence condition, and obtain the recognition model.

[0134] This application embodiment performs image recognition on each frame of video data using a recognition model to obtain the category of the frame image, thereby achieving automatic recognition of key frame images, further improving the recognition efficiency of key frame images, and effectively improving the accuracy of video monitoring.

[0135] This application provides a possible implementation method, wherein the preset playback conditions include at least one of the following:

[0136] Within the preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with the preset target decryption information.

[0137] Count the number of decrypted information that matches the target decrypted information, and ensure that the proportion of this number in the total number of keyframe images in the video data is not less than a preset threshold.

[0138] In this embodiment of the application, the monitoring of an advertising video is used as an example for specific explanation. In the advertising video, it can be set that when the decryption information of 6 consecutive keyframe images is consistent with the target decryption information, the advertising video continues to play; or it can be set that if the probability that the decryption information of all keyframe images in the advertising video is consistent with the target decryption information is more than 90%, the advertising video continues to play.

[0139] To better understand the video processing methods described above, the following section will combine... Figure 6 A detailed example of a video processing method according to this application, applied to a video monitoring server 601, is described. The method includes the following steps:

[0140] S601, randomly extract at least one original frame image from the original video.

[0141] S602, add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image.

[0142] The aforementioned outer border area can be the area corresponding to the BM area of ​​the liquid crystal display screen. This outer border area does not overlap with the original frame image and is within the display area of ​​the screen.

[0143] S603 replaces the corresponding original frame image with the keyframe image to generate video data, and sends the video data to the display screen 602 for playback.

[0144] S604 acquires a set of target frame images from the playing video data using a high-speed camera 603.

[0145] The aforementioned set of target frame images can be a subset of the frame images in the video data, or it can be all the frame images in the video data.

[0146] S605, based on a pre-trained recognition model, identifies a set of target frame images and obtains the category of each target frame image.

[0147] The category is used to characterize whether a cryptographic marker exists in the frame image. The above recognition model can be a model based on machine learning algorithms.

[0148] S606, when the category indicates that there is an encryption marker in the target frame image, the target frame image is used as the keyframe image.

[0149] S607 decrypts the encrypted markers of each keyframe image to obtain the decrypted information.

[0150] S608, compare the decryption information with the preset target decryption information. If the decryption information matches the target decryption information, the decryption result of the keyframe image is determined to be correct; if the decryption information does not match the target decryption information, or if the decryption information indicates that decryption is not possible, the decryption result of the keyframe image is incorrect.

[0151] S609, if the decryption result of at least one keyframe image is incorrect, control the display screen 602 to stop playing the video data; otherwise, continue playing the video data.

[0152] This application embodiment identifies at least one keyframe image from the playing video data and decrypts the encryption marker of each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, the video data continues to play; otherwise, playback stops. The video data is generated based on the encryption of the original video, and the keyframe image is a frame image in the video data containing the encryption marker. This application embodiment achieves encryption and monitoring of the original video based on keyframe images, i.e., single frames. This differs from existing technologies that directly overlay marker images over a large area and for a long time in the original video frame for video monitoring. This application obtains decryption information by decrypting keyframe images and then determines whether to continue playing the video data based on the decryption information. Since the display frequency of a single frame in the video data is much lower than the refresh rate of the human eye, keyframe images are not easily perceived by the human eye, effectively improving the integrity and smoothness of the video display and enhancing the user experience.

[0153] This application provides a video processing apparatus, such as... Figure 7 As shown, the video processing device 70 may include: an identification module 701, a decryption module 702, and a playback control module 703;

[0154] The identification module 701 is used to identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; and the keyframe image is a frame image in the video data that contains an encryption marker.

[0155] The decryption module 702 is used to decrypt the encrypted marker for each keyframe image to obtain the decryption information corresponding to the encrypted marker;

[0156] The playback control model 703 is used to continue playing video data when the decrypted information meets the preset playback conditions; otherwise, it stops playing video data.

[0157] This application provides a possible implementation method in which the above video data is encrypted and generated based on the following method:

[0158] Extract raw frame images from the original video;

[0159] Add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image;

[0160] The original frame images are replaced with keyframe images to generate video data.

[0161] This application provides a possible implementation method in which the above video data is encrypted and generated based on the following method:

[0162] Obtain the preset encrypted image;

[0163] Based on a preset time interval and timestamp, encrypted images are inserted as keyframe images into the original video to obtain video data.

[0164] This application embodiment provides a possible implementation method in which the recognition module 701, when recognizing at least one keyframe image, is used to:

[0165] The outer border region of each frame of the video data is detected, and when an encrypted marker is identified, the frame image is determined as a keyframe image.

[0166] This application embodiment provides a possible implementation method in which the recognition module 701, when recognizing at least one keyframe image, is used to:

[0167] Based on time intervals and timestamps, frame images are extracted from video data as keyframe images.

[0168] This application embodiment provides a possible implementation method in which the recognition module 701, when recognizing at least one keyframe image, is used to:

[0169] Based on a pre-trained recognition model, each frame of the video data is identified to obtain the category of the frame image; whereby the category is used to characterize whether there is an encrypted tag in the frame image.

[0170] If the category indicates that a cryptographic marker is present in the frame image, then the frame image is used as the keyframe image;

[0171] The recognition model was trained in the following way:

[0172] Obtain a preset sample image and its label category; the label category is used to characterize whether there is an encrypted marker in the sample image.

[0173] Input the sample image into the preset initial model to obtain the sample category in real time;

[0174] The parameters of the initial model are updated based on a preset loss function until the difference between the sample class and the label class meets the preset convergence condition, thus obtaining the recognition model.

[0175] This application provides a possible implementation method, wherein the preset playback conditions include at least one of the following:

[0176] Within the preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with the preset target decryption information.

[0177] The number of decrypted information messages that match the target decrypted information is counted, and the proportion of this number in the total number of keyframe images in the video data is not less than a preset threshold. The apparatus in this application embodiment can execute the method provided in this application embodiment, and their implementation principles are similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0178] This application embodiment identifies at least one keyframe image from the playing video data and decrypts the encryption marker of each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, the video data continues to play; otherwise, playback stops. The video data is generated based on the encryption of the original video, and the keyframe image is a frame image in the video data containing the encryption marker. This application embodiment achieves encryption and monitoring of the original video based on keyframe images, i.e., single frames. This differs from existing technologies that directly overlay marker images over a large area and for a long time in the original video frame for video monitoring. This application obtains decryption information by decrypting keyframe images and then determines whether to continue playing the video data based on the decryption information. Since the display frequency of a single frame in the video data is much lower than the refresh rate of the human eye, keyframe images are not easily perceived by the human eye, effectively improving the integrity and smoothness of the video display and enhancing the user experience.

[0179] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a video processing method. Compared with related technologies, this application can achieve the following: This application identifies at least one keyframe image from the playing video data and decrypts the encryption marker of each keyframe image to obtain corresponding decryption information. When the decryption information meets preset playback conditions, the video data continues to play; otherwise, the video data stops playing. The video data is generated based on the encryption of the original video, and the keyframe image is a frame image in the video data containing encryption markers. This application implements encryption and monitoring of the original video based on keyframe images, i.e., single frames. This differs from the prior art where marker images are directly superimposed on the original video screen over a large area for a long time for video monitoring. This application obtains decryption information by decrypting keyframe images, and then determines whether to continue playing the video data based on the decryption information. Since the display frequency of a single frame in the video data is much lower than the refresh rate of the human eye, keyframe images are not easily perceived by the human eye, effectively improving the integrity and smoothness of the video display and enhancing the user experience.

[0180] In one alternative embodiment, an electronic device is provided, such as Figure 8 As shown, Figure 8 The illustrated electronic device 800 includes a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, for example, via a bus 802. Optionally, the electronic device 800 may further include a transceiver 804, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 804 is not limited to one type, and the structure of this electronic device 800 does not constitute a limitation on the embodiments of this application.

[0181] Processor 801 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 801 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0182] Bus 802 may include a pathway for transmitting information between the aforementioned components. Bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 802 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0183] The memory 803 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0184] The memory 803 is used to store computer programs that execute the embodiments of this application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer programs stored in the memory 803 to implement the steps shown in the foregoing method embodiments.

[0185] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, and tablets, as well as fixed terminals such as digital TVs and desktop computers.

[0186] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0187] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following actions:

[0188] Identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; and the keyframe image is a frame image in the video data that contains encryption markers.

[0189] For each keyframe image, the encrypted marker is decrypted to obtain the decrypted information corresponding to the encrypted marker;

[0190] If the decrypted information meets the preset playback conditions, continue playing the video data; otherwise, stop playing the video data.

[0191] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0192] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0193] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A video processing method, characterized in that, include: Identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; the keyframe image is a frame image in the video data that contains an encryption marker, the encryption marker including at least one of a QR code, a barcode, and an encoded string; For each keyframe image, the encryption marker is decrypted to obtain the decryption information corresponding to the encryption marker; When the decryption information meets the preset playback conditions, the video data continues to play; otherwise, the video data stops playing. The preset playback conditions include at least one of the following: within a preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with the preset target decryption information; the number of decryption information consistent with the target decryption information is counted, and the proportion of the number of keyframe images in the video data is not less than a preset threshold. The identification of at least one keyframe image includes: Based on a pre-trained recognition model, each frame of the video data is identified to obtain the category of the frame image; wherein, the category is used to characterize whether there is an encrypted tag in the frame image; When the category indicates that an encryption marker is present in the frame image, the frame image is designated as a keyframe image. The recognition model is trained in the following manner: Obtain a preset sample image and its label category; wherein the label category is used to characterize whether there is an encrypted marker in the sample image; The sample image is input into a preset initial model to obtain the sample category output in real time; The parameters of the initial model are updated based on a preset loss function until the difference between the sample category and the label category satisfies a preset convergence condition, thus obtaining the recognition model.

2. The method according to claim 1, characterized in that, The video data was generated using encryption in the following manner: Extract the original frame images from the original video; Add encryption markers to the outer border area of ​​the original frame image to obtain the keyframe image; The video data is generated by replacing the corresponding original frame image with the keyframe image.

3. The method according to claim 1, characterized in that, The video data was generated using encryption in the following manner: Obtain the preset encrypted image; Based on a preset time interval and timestamp, the encrypted image is inserted as a keyframe image into the original video to obtain the video data.

4. The method according to claim 2, characterized in that, The identification of at least one keyframe image includes: The outer border region of each frame of the video data is detected, and when the encryption mark is identified, the frame image is determined as a keyframe image.

5. The method according to claim 3, characterized in that, The identification of at least one keyframe image includes: Based on the time interval and timestamp, frame images are extracted from the video data as keyframe images.

6. A video processing apparatus, characterized in that, include: The identification module is used to identify at least one keyframe image from the video data being played; wherein the video data is generated based on the encryption of the original video; the keyframe image is a frame image in the video data that contains an encryption mark, the encryption mark including at least one of a QR code, a barcode, and an encoded string; The decryption module is used to decrypt the encryption mark for each keyframe image to obtain the decryption information corresponding to the encryption mark; A playback control model is used to continue playing the video data when the decryption information meets preset playback conditions; otherwise, to stop playing the video data. The preset playback conditions include at least one of the following: within a preset playback time period, the decryption information corresponding to at least one keyframe image identified is consistent with preset target decryption information; the number of decryption information that is consistent with the target decryption information is counted, and the proportion of the number of such information to the total number of keyframe images in the video data is not less than a preset threshold. The recognition module is also used to recognize each frame of the video data based on a pre-trained recognition model to obtain the category of the frame image; wherein the category is used to characterize whether there is an encryption tag in the frame image; When the category indicates that an encryption marker is present in the frame image, the frame image is designated as a keyframe image. The recognition model is trained in the following manner: Obtain a preset sample image and its label category; wherein the label category is used to characterize whether there is an encrypted marker in the sample image; The sample image is input into a preset initial model to obtain the sample category output in real time; The parameters of the initial model are updated based on a preset loss function until the difference between the sample category and the label category satisfies a preset convergence condition, thus obtaining the recognition model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images