Video processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a video processing method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first video code stream corresponding to a target image sequence, wherein the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence; determining a second video code stream corresponding to the target image sequence, the second video code stream being obtained by performing first coding on a first object area in each frame of corresponding target image and performing second coding on a second object area in each frame of corresponding target image and then performing encryption, the code rate of the first object area after the first coding is greater than the code rate of the second object area after the second coding; and sending the first video code stream and the second video code stream from different transmission channels. According to the scheme, the network overhead during transmission of the video code stream can be greatly reduced, meanwhile, the video with privacy protection can still be previewed, and controllable display of sensitive information is achieved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video processing method, device, electronic equipment and storage medium. Background Art

[0002] With the development of the video security industry, the pursuit of higher quality visual experience has led to an overall improvement in image clarity, but it also makes the security protection of sensitive information particularly important.

[0003] In the relevant scheme, on the one hand, you can choose to protect the sensitive information in the video, but the recovery of sensitive information is a big difficulty, and it is not safe to transmit sensitive information in the same bit stream; on the other hand, you can choose to encrypt the entire video, but this will result in a large amount of data, too much performance overhead, and the encrypted data cannot be played and previewed. Summary of the invention

[0004] The present invention provides a video processing method, device, electronic device and storage medium to solve the problem of uncontrollable sensitive information in video code stream and uncontrollable network overhead.

[0005] According to one aspect of the present invention, a video processing method is provided, the method comprising:

[0006] Determine a first video code stream corresponding to a target image sequence, wherein the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence;

[0007] Determine a second video code stream corresponding to the target image sequence, wherein the second video code stream is obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the second object region, wherein a code rate of the first object region after the first encoding is greater than a code rate of the second object region after the second encoding;

[0008] The first video code stream and the second video code stream are sent from different transmission channels.

[0009] According to another aspect of the present invention, a video processing device is provided, the method comprising:

[0010] A first determination module is used to determine a first video code stream corresponding to a target image sequence, where the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence;

[0011] A second determination module is used to determine a second video code stream corresponding to the target image sequence, wherein the second video code stream is obtained by first encoding a first object region in each frame corresponding to the target image and second encoding a second object region in each frame corresponding to the target image and then encrypting the resultant, wherein a code rate of the first object region after the first encoding is greater than a code rate of the second object region after the second encoding;

[0012] The transmission module is used to send the first video code stream and the second video code stream from different transmission channels.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video processing method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video processing method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is to separate the first object area and the second object area in the target image area corresponding to each frame in the target image sequence, scramble the first object area in the target image corresponding to each frame in the target image sequence to obtain a first video code stream, perform a first encoding on the first object area in the target image corresponding to each frame and a second encoding on the second object area in the target image corresponding to each frame and then encrypt to obtain a second video code stream, after object separation, the first object area and the second object area are scrambled and encrypted independently encoded respectively, and the encoding parameters are adjusted according to the business so that the encoding bit rate of the second object area side is lower than the encoding bit rate of the first object area, even if two channels of video code streams are used, the network overhead when the video code stream is transmitted can be greatly reduced, and at the same time, because the first object area with sensitive information is scrambled, the video with privacy protection can still be previewed, and the sensitive information can be controlled. Display.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a video processing method provided according to an embodiment of the present invention;

[0022] Figure 2 It is a schematic diagram of a process for performing dual-channel encoding on an image sequence to form a video code stream according to an embodiment of the present invention;

[0023] Figure 3 is a flowchart of another video processing method provided according to an embodiment of the present invention

[0024] Figure 4 is a structural schematic diagram of a video processing device provided according to an embodiment of the present invention;

[0025] Figure 5 It is a structural schematic diagram of an electronic device for implementing the video processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Figure 1A flowchart of a video processing method is provided for an embodiment of the present invention. This embodiment can be applicable to the case of performing scrambling and encryption mixed coding processing on an image sequence in a video acquired by a shooting device. The method can be executed by a video processing device. The video processing device can be implemented in the form of hardware and / or software. The video processing device can be configured in any electronic device with a network communication function, and the electronic device can be a camera, a terminal or a server, etc.

[0029] like Figure 1 As shown, the video processing method in this embodiment may include the following process:

[0030] S110, determining a first video code stream corresponding to the target image sequence, where the first video code stream is obtained by scrambling a first object region in the target image corresponding to each frame of the target image sequence.

[0031] The target image sequence may be an image sequence consisting of the corresponding frames of the video to be encoded or an image sequence consisting of multiple frames of images selected from the corresponding frames of the video to be encoded. The video to be encoded may be a video clip obtained by capturing images of the detection area through a camera and transmitted.

[0032] For each frame in the target image sequence corresponding to the target image, the target image includes a first object area and a second object area, the first object area includes at least a part of the target object, and the second object area does not include the part of the target object, and the target object can be a pre-configured object that needs to be encrypted and protected. For example, the target object can be a pre-configured vehicle, person or other object in the image that needs to be encrypted and protected.

[0033] Since the target image corresponding to the target image sequence includes the first object area, it is not easy to preview the target image sequence when it is displayed after being transmitted based on the target image sequence in the form of a video code stream, otherwise the first object area containing sensitive information in the target image will be leaked, causing security risks.

[0034] For this purpose, see Figure 2 After separating the first object area and the second object area in the target image, the first object area in the target image is scrambled. After scrambling, the clear first object area cannot be seen from the target image. Only the blurred first object area in the target image and the clear second object area used to preview the scene area where the image is located can be seen. Then, after scrambling the first object area, the first video code stream corresponding to the target image sequence is obtained.

[0035] As an optional but non-limiting implementation, before determining the first video code stream corresponding to the target image sequence, the following steps A1-A3 may also be included:

[0036] Step A1: divide the target image corresponding to each frame of the target image sequence into blocks to obtain a plurality of block areas.

[0037] Step A2: Detect the position of the target object in the target image corresponding to each frame, and determine the block area corresponding to the target object position in the target image as the first object area and mark the area position.

[0038] Step A3: determine the remaining block areas in the target image except the first object area as the second object area and mark the area positions.

[0039] See also Figure 2 For each frame of the target image sequence corresponding to the target image, the target image is adaptively divided into blocks to obtain multiple block areas of the same size, for example, the default block area size can be 8*8 pixels. The position of the target object in each frame of the target image is determined, recorded as the target object position, for example, a target object recognition model can be used to perform feature recognition on the target image, and the target object and the position of the target object can be identified from the target image.

[0040] Optionally, for multiple consecutive frames of target images, the same target object included in each frame of the target image can be considered to be continuous in the motion process, and the optical flow method can be used to perform position recognition on the same target object included in the multiple consecutive frames of target images to obtain the target object position of the same target object in the target image of each frame.

[0041] See also Figure 2 After detecting the position of the target object in each frame corresponding to the target image and determining the first object area in the target image, the first object area is marked. According to the area position marking of the first object area in the target image, the first object area and the second object area can be separated from the target image. The target image can include two parts, namely the first object area and the second object area separated from the target image. The first object area at least includes a part of the target object, which is recorded as area 1, and the second object area does not include a part of the target object, which is recorded as area 2. The two areas can be combined to form the target image.

[0042] Optionally, the target image corresponding to each frame of the target image sequence can adopt an IPPP structure, and the target image corresponding to each frame of the target image sequence is processed in real time. The image frame rate between the target images corresponding to each frame of the target image sequence can be 25fps, and the interval between the target images corresponding to each frame to be processed is 40ms. The result is updated once every period T, where the period T can be 200ms.

[0043] S120, determining a second video code stream corresponding to the target image sequence, the second video code stream being obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the resultant, the code rate of the first object region after the first encoding being greater than the code rate of the second object region after the second encoding.

[0044] For the shooting scenes of cameras and other shooting devices, although part of the scene content such as the first object area in the target image needs to be encrypted and protected, it does not mean that this part of the first object area will still need to be displayed in the future. At the same time, considering that the first object area with sensitive information may have security risks if it is placed in the same bitstream for transmission. In this way, an image encoding channel can be configured to encrypt and protect the first object area in the target image, so that the video bitstream encrypted for the first object area in the target image can be encoded and then the encrypted first object area can be decrypted and displayed according to the authority.

[0045] However, if the first object area in the target image is simply encrypted to form a video stream, then combined with the video stream formed by the previous scrambling, it can be considered that it is equivalent to directly copying the target image to form two channel video streams. The video stream formed by simple encryption will increase network bandwidth overhead and generate additional redundancy. In addition, there are multiple first object areas in the target image, and encryption for each first object area requires multiple unnecessary encryption operations.

[0046] For this purpose, see Figure 2, the first object area and the second object area in the target image can be independently encoded, and the encoding bit rate used in the second encoding process of the second object area is guaranteed to be lower than the encoding bit rate used in the first encoding process of the first object area, and the bit rate of the second object area after the second encoding is lower than the original bit rate of the second object area, so that by reducing the bit rate when encoding the second object area, the network bandwidth overhead of the formed video code stream during transmission and the calculation overhead of the formed video code stream are reduced to a certain extent, so that the network bandwidth overhead of the video code stream can be controlled. At the same time, after the first object area and the second object area in the target image are independently encoded, the video code streams corresponding to the independently encoded target image sequence are uniformly encrypted to obtain the first video code stream corresponding to the target image sequence, so that the sensitive information in the target image can be controlled.

[0047] S130: Send the first video code stream and the second video code stream from different transmission channels.

[0048] See also Figure 2 , the first object area of ​​the target image corresponding to each frame in the first video code stream is scrambled, while the second object area is not scrambled. The first video code stream is displayed independently to achieve a safe preview of the target image. The first object area and the second object area of ​​each frame corresponding to the target image in the second video code stream are encoded at different rates to ensure that the network bandwidth overhead occupied by the generated video code stream is controllable. At the same time, the encryption of the first object area can ensure that sensitive information is controllable. After the second video code stream is decrypted and displayed, the first object area can be clearly displayed. When both the first object area and the second object area need to be clearly displayed, the second object area of ​​the target image corresponding to each frame in the first video code stream and the first object area of ​​the target image corresponding to each frame in the second video code stream can be decrypted and then fused for display. In the above manner, different video display requirements can be met. For example, some scenes require previewing videos after privacy protection, some scenes require displaying videos in full detail, and some scenes only focus on local video areas with target objects.

[0049] The technical solution of the embodiment of the present invention is to separate the first object area and the second object area in the target image area corresponding to each frame in the target image sequence, scramble the first object area in the target image corresponding to each frame in the target image sequence to obtain a first video code stream, perform a first encoding on the first object area in the target image corresponding to each frame and a second encoding on the second object area in the target image corresponding to each frame and then encrypt to obtain a second video code stream, after object separation, the first object area and the second object area are scrambled and encrypted independently encoded respectively, and the encoding parameters are adjusted according to the business so that the encoding bit rate of the second object area side is lower than the encoding bit rate of the first object area, even if two channels of video code streams are used, the network overhead when the video code stream is transmitted can be greatly reduced, and at the same time, because the first object area with sensitive information is scrambled, the video with privacy protection can still be previewed, and the sensitive information can be controlled. Display.

[0050] Figure 3 A flowchart of another video processing method is provided for an embodiment of the present invention. The technical solution of this embodiment further optimizes the process of determining the first video code stream corresponding to the target image sequence in the aforementioned embodiment on the basis of the aforementioned embodiment. This embodiment can be combined with various optional solutions in one or more of the aforementioned embodiments.

[0051] like Figure 3 As shown, the video processing method in this embodiment may include the following process:

[0052] S310 , in a process of performing a third encoding on a first object region in a target image corresponding to each frame included in a target image sequence, scrambling the first object region in the target image.

[0053] Optionally, the first object area and the second object area are block areas obtained by performing target object detection on the target image corresponding to each frame in sequence, the first object area is a block area corresponding to the target image that contains a local part of the target object, and the second object area is a block area corresponding to the target image that does not contain a local part of the target object. The block area corresponding to the target image is determined by dividing the target image into blocks, and the target object can be determined by performing target detection on the target image or an object in a preset fixed area in the target image.

[0054] As an optional but non-limiting implementation, in the process of performing the third encoding on the first object area in the target image corresponding to each frame, scrambling the first object area may include steps B1-B3:

[0055] Step B1: predicting a pixel block in a first object area in a target image based on an intra-frame prediction mode to obtain a prediction residual, where the prediction residual is a difference between an original pixel block and a predicted pixel block.

[0056] Step B2: performing discrete cosine transform on the preset residual corresponding to the first object area, and performing scrambling processing on the prediction residual of the discrete cosine transform.

[0057] Step B3: Perform a scrambling process on the prediction residual of the discrete cosine transform and then perform a third encoding.

[0058] See also Figure 2 In the first coding channel, the prediction mode of the target image is configured as the intra-frame prediction mode, and then the original pixel block of the first object area in the target image is predicted using the intra-frame prediction to obtain the predicted pixel block, and the difference between the original pixel block and the predicted pixel block of the first object area is calculated to obtain the predicted residual. The preset residual corresponding to the first object area is subjected to discrete cosine transform (DCT transform) to obtain the predicted residual after DCT transform. On this basis, the predicted residual after DCT transform is scrambled, and the predicted residual after scrambling is used for third encoding, and the third encoding can be entropy encoding. Optionally, for the relationship between the macroblock culling of the first object area in the target image and the first object area in the target object corresponding to the subsequent frame, the influence transmission of the macroblock between the first object area of ​​the target image corresponding to each frame is reduced.

[0059] By adopting the above optional scheme, in the first encoding channel, transform domain scrambling processing can be performed on the first object area in the target image, and then the first object area can be encoded to implement scrambling processing on the first object area to avoid leakage of sensitive information of the first object area, thereby achieving controllability of sensitive information of the first object area.

[0060] S320, performing fourth encoding on the second object region in the target image corresponding to each frame, the first object region is a region including a part of the target object, and the second object region is a region not including the part of the target object.

[0061] As an optional but non-limiting implementation, performing fourth encoding on the second object area in the target image corresponding to each frame may include steps C1-C2:

[0062] Step C1: predicting a pixel block of a second object area in a target image based on an intra-frame prediction mode or an inter-frame prediction mode to obtain a prediction residual, where the prediction residual is a difference between an original pixel block and a predicted pixel block.

[0063] Step C2: performing discrete cosine transform on the preset residual corresponding to the second object area, and performing fourth encoding on the prediction residual of the discrete cosine transform.

[0064] See also Figure 2In the first coding channel, the prediction mode of the target image can also be configured as an intra-frame prediction mode or an inter-frame prediction mode, and then the intra-frame prediction mode or the inter-frame prediction mode is selected to predict the original pixel block of the second object area in the target image to obtain a predicted pixel block, and the difference between the original pixel block and the predicted pixel block of the second object area is calculated to obtain a predicted residual. The preset residual corresponding to the second object area can be discrete cosine transformed (DCT transformed) to obtain a predicted residual after DCT transformation, and then the predicted residual after DCT transformation is directly subjected to fourth encoding, and the fourth encoding can be entropy coding.

[0065] S330: Determine a first video code stream corresponding to the target image sequence based on a third encoding result of the first object region and a fourth encoding result of the second object region in the target image corresponding to each frame.

[0066] The first video code stream is obtained by scrambling the first object area in the target image corresponding to each frame of the target image sequence.

[0067] After encoding the first object area in the target image in the first encoding channel to obtain the third encoding result and encoding the second object area to obtain the fourth encoding result, a target image having the third encoding result and the fourth encoding result can be obtained, and a code stream is generated using the target image having the third encoding result and the fourth encoding result corresponding to each frame to obtain a first video code stream corresponding to the target image sequence.

[0068] S340, determining a second video code stream corresponding to the target image sequence, the second video code stream being obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the resultant, the code rate of the first object region after the first encoding being greater than the code rate of the second object region after the second encoding.

[0069] As an optional but non-limiting implementation, determining the second video code stream corresponding to the target image sequence may include steps D1-D3:

[0070] Step D1, performing first encoding on a first object region in a target image corresponding to each frame, wherein the first encoding adjusts the first object region from an original bit rate to a first bit rate after encoding the first object region, and the original bit rate is less than or equal to the first bit rate.

[0071] Step D2, performing a second encoding on the second object area in the target image corresponding to each frame, adjusting the first object area from the original bit rate to the second bit rate after encoding the second object area with the first encoding, the original bit rate is greater than the second bit rate, the first object area is an area including a local part of the target object, and the second object area is an area not including a local part of the target object.

[0072] Step D3, obtaining a third video code stream corresponding to the target image sequence based on the first encoding result of the first object area and the second encoding result of the second object area in each frame corresponding to the target image, and encrypting the third video code stream to obtain a second video code stream.

[0073] See also Figure 2 In the second coding channel, the first object area and the second object area in the target image are encoded independently. After encoding the first object area, the adjusted first code rate of the first object area will not be less than the original code rate of the first object area, so as to achieve high-quality encoding for the first object area; after encoding the second object area in the target image, the adjusted second code rate of the first object area can be made less than the original code rate of the first object area, so as to achieve low-quality encoding of the second object area, reduce the network bandwidth overhead of the second object area in the target image during network transmission, and the second object area is used as an unimportant area to save bandwidth as much as possible without transmitting residuals. All P frame areas are encoded in Skip mode to reduce the code rate as much as possible. Optionally, the code rate to be used for the second object area can be adaptively configured according to business needs.

[0074] After encoding the first object area in the target image to obtain the first encoding result and encoding the second object area to obtain the second encoding result in the second encoding channel, a target image having the first encoding result and the second encoding result can be obtained, and a code stream is generated using the target image having the first encoding result and the second encoding result corresponding to each frame to obtain a third video code stream corresponding to the target image sequence, and then the third video code stream is encrypted to obtain the second video code stream.

[0075] Optionally, after the second channel encoding process, the first video code stream is obtained by encrypting the third video code stream to ensure code stream security. At the same time, the key used for encrypting the third video code stream is stored in the supplementary enhancement information of the target image for carrying.

[0076] S350: Send the first video code stream and the second video code stream from different transmission channels.

[0077] As an optional but non-limiting implementation, before sending the first video code stream and the second video code stream from different transmission channels, steps E1-E2 are further included:

[0078] Step E1, adding first object region parameter content of each frame corresponding to the target image to the supplementary enhancement information of each frame corresponding to the target image included in the first video code stream, wherein the first object region parameter content includes the number, position and size of the first object region in the target image.

[0079] Step E2: Add the first object region parameter content of each frame corresponding to the target image to the supplementary enhancement information of each frame corresponding to the target image included in the second video code stream, wherein the first object region parameter content includes the number, position and size of the first object region in the target image.

[0080] The first object area parameter is used to replace the first object area in the first video stream with the first object area in the second video stream when the first video stream and the second video stream are fused and displayed.

[0081] See also Figure 2 , the first object area parameter may include the number NUM of the first object areas separated from the target image, and the vertex coordinates (x, y) and length and width parameters (W, H) of the block area corresponding to a single first object area. Of course, each block area may also be an irregularly shaped area, rather than a regular shaped area of ​​a fixed size. The first object area parameter is carried in the supplementary enhancement information (SEI) before each frame of the target image in the first video stream and the encrypted second video stream. The first object area parameter can be used to align the first object area in the first video stream with the first object area in the second video stream and replace them after the block areas are aligned when the first video stream and the second video stream are fused and displayed.

[0082] As an optional but non-limiting implementation, before sending the first video code stream and the second video code stream from different transmission channels, steps F1-F2 are also included:

[0083] Step F1: adding the image time information of the target image corresponding to each frame included in the first video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the first video code stream.

[0084] Step F1: adding the image time information of the target image corresponding to each frame included in the second video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the second video code stream.

[0085] The image time information in the supplementary enhancement information of the key frame is used to synchronize the target image corresponding to each frame included in the first video code stream with the target image corresponding to each frame included in the second video code stream.

[0086] See also Figure 2, for each target image corresponding to each frame included in the first video code stream, image time information of the target image is obtained. The image time information is an image timestamp used to mark the time point of target image acquisition, and the image timestamp in the image time information is converted into UTC time and carried in the supplementary enhancement information SEI of the key frame in each frame corresponding to the target image, and the image time information is used to synchronize the target image corresponding to each frame included in the first video code stream with the target image corresponding to each frame included in the second video code stream, and the target images corresponding to subsequent frames can be synchronized according to the image time information carried in the key frame.

[0087] Optionally, in the case where the second object area of ​​the target image corresponding to each frame in the first video code stream and the first object area of ​​the target image corresponding to each frame in the second video code stream are decrypted and then fused and displayed, a key frame containing image time information can be determined from the target image corresponding to each frame, and the image time information in the key frame can be used to synchronize and align the target images corresponding to each frame in the first video code stream and the second video code stream, and the second video code stream is decrypted to obtain a decrypted video code stream, and then the first object area of ​​the target image corresponding to each frame in the decrypted second video code stream replaces the first object area of ​​the target image corresponding to each frame in the first video code stream, so as to achieve fused display of the first video code stream and the second video code stream.

[0088] The technical solution of the embodiment of the present invention extracts and separates the first object area and the second object area through an intelligent module, uses image timestamps for synchronization, combines the IPC multi-stream characteristics, adopts dual-channel separation of the first object area and the second object area, and then performs mixed encoding of encryption code stream and scrambled code stream in sequence, adjusts encoding parameters according to the business, carries synchronization information and block area location information with the video code stream, displays video content according to user permissions at the decoding end, adds image timestamps, and area marking information for auxiliary encoding and decoding, and uses encryption code streams to reconstruct scrambled areas, which can ensure that sensitive monitoring information is controllable and network overhead is as small as possible.

[0089] Figure 4 A structural block diagram of a video processing device is provided for an embodiment of the present invention. This embodiment can be applied to the case of performing scrambling and encryption mixed coding processing on an image sequence in a video acquired by a shooting device. The video processing device can be implemented in the form of hardware and / or software. The video processing device can be configured in any electronic device with a network communication function, which can be a camera, a terminal or a server, etc.

[0090] like Figure 4 As shown, the video processing device in this embodiment may include: a first determination module 410, a second determination module 420 and a transmission module 430. Among them:

[0091] A first determination module 410 is used to determine a first video code stream corresponding to a target image sequence, wherein the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence;

[0092] A second determination module 420 is used to determine a second video code stream corresponding to the target image sequence, wherein the second video code stream is obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the two regions, wherein a code rate of the first object region after the first encoding is greater than a code rate of the second object region after the second encoding;

[0093] The transmission module 430 is configured to send the first video code stream and the second video code stream through different transmission channels.

[0094] Based on the above embodiment, optionally, determining a first video code stream corresponding to the target image sequence includes:

[0095] In the process of performing third encoding on the first object area in the target image corresponding to each frame, scrambling the first object area;

[0096] Performing fourth encoding on a second object region in a target image corresponding to each frame, wherein the first object region is a region including a part of the target object, and the second object region is a region not including a part of the target object;

[0097] Based on the third encoding result of the first object area and the fourth encoding result of the second object area in the target image corresponding to each frame, a first video code stream corresponding to the target image sequence is determined.

[0098] Based on the above embodiment, optionally, the first object area is a block area corresponding to the target image that contains a local target object, and the second object area is a block area corresponding to the target image that does not contain a local target object, and the block area corresponding to the target image is determined by dividing the target image into blocks.

[0099] Based on the above embodiment, optionally, in the process of performing third encoding on the first object area in the target image corresponding to each frame, scrambling the first object area includes:

[0100] Predicting a pixel block of a first object area in a target image based on an intra-frame prediction mode to obtain a prediction residual, wherein the prediction residual is a difference between an original pixel block and a predicted pixel block;

[0101] Performing discrete cosine transform on a preset residual corresponding to the first object area, and performing scrambling processing on a prediction residual of the discrete cosine transform;

[0102] The prediction residual of the discrete cosine transform is scrambled and then thirdly encoded.

[0103] Based on the above embodiment, optionally, performing fourth encoding on the second object area in the target image corresponding to each frame includes:

[0104] Predicting a pixel block of a second object area in the target image based on an intra-frame prediction mode or an inter-frame prediction mode to obtain a prediction residual, where the prediction residual is a difference between an original pixel block and a predicted pixel block;

[0105] A discrete cosine transform is performed on a preset residual corresponding to the second object area, and a fourth encoding is performed on the prediction residual of the discrete cosine transform.

[0106] Based on the above embodiment, optionally, determining a second video code stream corresponding to the target image sequence includes:

[0107] Performing a first encoding on a first object region in a target image corresponding to each frame, wherein the first encoding adjusts the first object region from an original bit rate to a first bit rate after encoding the first object region, and the original bit rate is less than or equal to the first bit rate;

[0108] Performing a second encoding on a second object region in a target image corresponding to each frame, wherein the first encoding adjusts the first object region from an original bit rate to a second bit rate after encoding the second object region, the original bit rate is greater than the second bit rate, the first object region is a region including a part of the target object, and the second object region is a region not including a part of the target object;

[0109] A third video code stream corresponding to the target image sequence is obtained based on a first encoding result of a first object area and a second encoding result of a second object area in each frame corresponding to the target image, and the third video code stream is encrypted to obtain a second video code stream.

[0110] Based on the above embodiment, optionally, before determining the first video code stream corresponding to the target image sequence, the method further includes:

[0111] Dividing the target image corresponding to each frame of the target image sequence into blocks to obtain a plurality of block areas;

[0112] Detecting the position of the target object in the target image corresponding to each frame, and determining the block area corresponding to the position of the target object in the target image as the first object area and marking the area position;

[0113] The remaining block areas in the target image except the first object area are determined as second object areas and area positions are marked.

[0114] Based on the above embodiment, optionally, before sending the first video code stream and the second video code stream from different transmission channels, the method further includes:

[0115] In the supplementary enhancement information of the target image corresponding to each frame included in the first video code stream, first object region parameter content corresponding to the target image of each frame is added, where the first object region parameter content includes the number, position and size of the first object region in the target image;

[0116] Adding, in the supplementary enhancement information of the target image corresponding to each frame included in the second video code stream, first object region parameter content corresponding to the target image of each frame, wherein the first object region parameter content includes the number, position and size of the first object region in the target image;

[0117] The first object area parameter is used to replace the first object area in the first video stream with the first object area in the second video stream when the first video stream and the second video stream are fused and displayed.

[0118] Based on the above embodiment, optionally, before sending the first video code stream and the second video code stream from different transmission channels, the method further includes:

[0119] Adding the image time information of the target image corresponding to each frame included in the first video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the first video code stream;

[0120] Adding the image time information of the target image corresponding to each frame included in the second video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the second video code stream;

[0121] The image time information in the supplementary enhancement information of the key frame is used to synchronize the target image corresponding to each frame included in the first video code stream with the target image corresponding to each frame included in the second video code stream.

[0122] The technical solution of the embodiment of the present invention is to separate the first object area and the second object area in the target image area corresponding to each frame in the target image sequence, scramble the first object area in the target image corresponding to each frame in the target image sequence to obtain a first video code stream, perform a first encoding on the first object area in the target image corresponding to each frame and a second encoding on the second object area in the target image corresponding to each frame and then encrypt to obtain a second video code stream, after object separation, the first object area and the second object area are scrambled and encrypted independently encoded respectively, and the encoding parameters are adjusted according to the business so that the encoding bit rate of the second object area side is lower than the encoding bit rate of the first object area, even if two channels of video code streams are used, the network overhead when the video code stream is transmitted can be greatly reduced, and at the same time, because the first object area with sensitive information is scrambled, the video with privacy protection can still be previewed, and the sensitive information can be controlled. Display.

[0123] The video processing device provided in the embodiment of the present invention can execute the video processing method provided in any embodiment of the present invention, and has the corresponding functions and beneficial effects of executing the video processing method. For detailed process, please refer to the relevant operations of the video processing method in the above embodiment.

[0124] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0125] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0126] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0127] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a video processing method.

[0129] In some embodiments, the video processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the video processing method in any other appropriate manner (e.g., by means of firmware).

[0130] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0132] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0134] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0135] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0136] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0137] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video processing method, characterized in that: The method comprises: Determine a first video code stream corresponding to a target image sequence, wherein the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence; Determine a second video code stream corresponding to the target image sequence, wherein the second video code stream is obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the second object region, wherein a code rate of the first object region after the first encoding is greater than a code rate of the second object region after the second encoding; The first video code stream and the second video code stream are sent from different transmission channels.

2. The method according to claim 1, characterized in that Determining a first video code stream corresponding to a target image sequence includes: In the process of performing third encoding on the first object area in the target image corresponding to each frame, scrambling the first object area; Performing fourth encoding on a second object region in a target image corresponding to each frame, wherein the first object region is a region including a part of the target object, and the second object region is a region not including a part of the target object; Based on the third encoding result of the first object area and the fourth encoding result of the second object area in the target image corresponding to each frame, a first video code stream corresponding to the target image sequence is determined.

3. The method according to claim 2, characterized in that The first object area is a block area corresponding to the target image that contains a local part of the target object, and the second object area is a block area corresponding to the target image that does not contain a local part of the target object. The block area corresponding to the target image is determined by dividing the target image into blocks.

4. The method according to claim 2, characterized in that: In the process of performing third encoding on the first object area in the target image corresponding to each frame, scrambling the first object area includes: Predicting a pixel block of a first object area in a target image based on an intra-frame prediction mode to obtain a prediction residual, wherein the prediction residual is a difference between an original pixel block and a predicted pixel block; Performing discrete cosine transform on a preset residual corresponding to the first object area, and performing scrambling processing on a prediction residual of the discrete cosine transform; The prediction residual of the discrete cosine transform is scrambled and then thirdly encoded.

5. The method according to claim 2, characterized in that: Performing fourth encoding on the second object area in the target image corresponding to each frame, comprising: Predicting a pixel block of a second object area in the target image based on an intra-frame prediction mode or an inter-frame prediction mode to obtain a prediction residual, where the prediction residual is a difference between an original pixel block and a predicted pixel block; A discrete cosine transform is performed on a preset residual corresponding to the second object area, and a fourth encoding is performed on the prediction residual of the discrete cosine transform.

6. The method according to claim 1, characterized in that Determining a second video code stream corresponding to the target image sequence includes: Performing a first encoding on a first object region in a target image corresponding to each frame, wherein the first encoding adjusts the first object region from an original bit rate to a first bit rate after encoding the first object region, and the original bit rate is less than or equal to the first bit rate; Performing a second encoding on a second object region in a target image corresponding to each frame, wherein the first encoding adjusts the first object region from an original bit rate to a second bit rate after encoding the second object region, the original bit rate is greater than the second bit rate, the first object region is a region including a part of the target object, and the second object region is a region not including a part of the target object; A third video code stream corresponding to the target image sequence is obtained based on a first encoding result of a first object area and a second encoding result of a second object area in each frame corresponding to the target image, and the third video code stream is encrypted to obtain a second video code stream.

7. The method according to any one of claims 1 to 6, characterized in that: Before determining the first video code stream corresponding to the target image sequence, the method further includes: Dividing the target image corresponding to each frame of the target image sequence into blocks to obtain a plurality of block areas; Detecting the position of the target object in the target image corresponding to each frame, and determining the block area corresponding to the position of the target object in the target image as the first object area and marking the area position; The remaining block areas in the target image except the first object area are determined as second object areas and area positions are marked.

8. The method according to any one of claims 1 to 6, characterized in that: Before sending the first video code stream and the second video code stream from different transmission channels, the method further includes: Adding, in the supplementary enhancement information of the target image corresponding to each frame included in the second video code stream, first object region parameter content corresponding to the target image of each frame, wherein the first object region parameter content includes the number, position and size of the first object region in the target image; The first object area parameter is used to replace the first object area in the first video stream with the first object area in the second video stream when the first video stream and the second video stream are fused and displayed.

9. The method according to any one of claims 1 to 6, characterized in that: Before sending the first video code stream and the second video code stream from different transmission channels, the method further includes: Adding the image time information of the target image corresponding to each frame included in the first video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the first video code stream; Adding the image time information of the target image corresponding to each frame included in the second video code stream to the supplementary enhancement information of the key frame in the target image corresponding to each frame included in the second video code stream; The image time information in the supplementary enhancement information of the key frame is used to synchronize the target image corresponding to each frame included in the first video code stream with the target image corresponding to each frame included in the second video code stream.

10. A video processing device, characterized in that: The method comprises: A first determination module is used to determine a first video code stream corresponding to a target image sequence, where the first video code stream is obtained by scrambling a first object region in a target image corresponding to each frame of the target image sequence; A second determination module is used to determine a second video code stream corresponding to the target image sequence, wherein the second video code stream is obtained by performing a first encoding on a first object region in each frame corresponding to the target image and performing a second encoding on a second object region in each frame corresponding to the target image and then encrypting the result, wherein a code rate of the first object region after the first encoding is greater than a code rate of the second object region after the second encoding; The transmission module is used to send the first video code stream and the second video code stream from different transmission channels.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video processing method according to any one of claims 1 to 9 when executed.

Images