Video coding system and method and electronic equipment

Abstract

The invention provides a video coding system and method and electronic equipment, and relates to the technical field of video coding, and the system comprises an image processing unit which is used for outputting a to-be-processed image, and the to-be-processed image is generated based on a frame of image in a target video; the online scaling unit is used for scaling the resolution of the to-be-processed image; the online information labeling unit is used for labeling the to-be-processed image processed by the online zooming unit so as to generate a first labeled image, and the first labeled image is labeled with motion information; the image marking unit is used for performing marking operation on the to-be-processed image to generate a second marked image, and the marking operation is used for marking a key area of the to-be-processed image; and an encoding unit for encoding the second tagged image based on the motion information in the first tagged image. According to the system provided by the invention, the video coding quality can be improved under the condition that the hardware resources of the video coding equipment are compact.

Description

Technical Field

[0001] The present application relates to the field of video encoding technology, and in particular to a video encoding system, method and electronic device. Background Art

[0002] In the field of general security, the installation space required for various cameras is generally narrow, so the hardware conditions are relatively compact. In order to better identify the target, the actual requirements of security, monitoring and other scenes are to have high requirements on the quality of dynamic images. Due to the lack of hardware equipment, the related cameras do not support video encoding in dynamic scenes, which will bring technical problems such as unclear image quality, tailing and discoloration. Summary of the invention

[0003] The embodiments of the present application provide a video encoding system, method and electronic device. In processing video encoding, motion information is marked on a smaller resolution version of each frame in the video, and the original image is encoded based on the image, thereby improving the video encoding quality when the hardware resources of the video encoding device are compact.

[0004] In a first aspect, an embodiment of the present application provides a video encoding system, including:

[0005] An image processing unit, the image processing unit is used to output an image to be processed, wherein the image to be processed is generated based on a frame of image in a target video;

[0006] An online scaling unit, connected to the image processing unit for communication, and configured to reduce the resolution of the image to be processed;

[0007] an online information labeling unit, which is in communication with the online scaling unit, and is used to label the image to be processed after being processed by the online scaling unit to generate a first labeled image, wherein the first labeled image is labeled with motion information;

[0008] an image marking unit, which is in communication with the image processing unit, and is used to perform a marking operation on the image to be processed to generate a second marked image, wherein the marking operation is used to mark a key area of ​​the image to be processed;

[0009] The encoding unit is respectively connected to the image marking unit and the online information tagging unit for communication, and the encoding unit is used to encode the second marked image based on the motion information in the first marked image.

[0010] In some embodiments, the image processing unit includes:

[0011] An image processing subunit, the image processing subunit being used to optimize a frame of image in the received target video to generate the image to be processed;

[0012] An online conversion subunit, wherein the online conversion subunit is respectively communicatively connected with the image processing subunit, the image marking unit and the online scaling unit, and the online conversion subunit is used to convert the image to be processed into images to be processed of at least two resolutions, and send the converted images to be processed of each resolution to the online scaling unit and the image marking unit.

[0013] In some embodiments, the number of the image marking unit, the online scaling unit, and the online information labeling unit is the same as the number of the at least two online conversion sub-units, and the image marking unit and the online scaling unit are both communicatively connected to the at least two online conversion sub-units one-to-one, and the online scaling unit is communicatively connected to the online information labeling unit one-to-one.

[0014] In some implementations, the online conversion subunit includes at least two online scalers, and the at least two online scalers are respectively connected to the image processing subunit and the online scaling unit by reading a hardware cache.

[0015] In some embodiments, the online scaling unit is a proportional scaler.

[0016] In some implementations, the video encoding system further includes:

[0017] An image signal processor is communicatively connected to the image processing unit, and is used for optimizing each frame of the received target video and sending the optimized image to the image processing unit.

[0018] In some embodiments, the online information tagging unit includes an online preset algorithm engine and an online image tagging subunit;

[0019] The online preset algorithm engine is respectively connected to the online scaling unit and the online image marking subunit for communication, and the online preset algorithm engine is used to mark motion information in the image to be processed after being processed by the online scaling unit;

[0020] The online image marking subunit is used to receive the image to be processed after being marked by the preset algorithm engine, and perform a marking operation on the image to be processed to generate the first marked image, wherein the marking operation is used to mark the key area of ​​the image to be processed.

[0021] In some implementations, the online scaling unit is connected to the image processing unit by reading a hardware cache;

[0022] The online scaling unit is connected to the online information marking unit by reading a hardware cache.

[0023] In a second aspect, an embodiment of the present application provides a video encoding method, used in the video encoding system as described in any one of the first aspects, the video encoding method comprising:

[0024] In response to an instruction from a target terminal, the image processing unit is controlled to convert a received image into the image to be processed, wherein the received image is each frame of an image in a target video;

[0025] Controlling the online scaling unit to scale the image to be processed, and sending the scaled image to be processed to the online information labeling unit;

[0026] Controlling the online information annotation unit to perform motion information annotation on the scaled image to be processed to generate the first labeled image;

[0027] Controlling the image marking unit to mark the key area of ​​the image to be processed to generate the second marked image;

[0028] The encoding unit is controlled to encode the second marked image based on the motion information in the first marked image to obtain an encoded image.

[0029] The encoded images are sent to the target terminal in the encoding order.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0031] Memory for storing computer programs;

[0032] A processor is used to execute the computer program to implement the steps of the method described in the second aspect.

[0033] In summary, the video encoding system provided by the embodiment of the present application reduces the resolution of each frame of the target video, marks the motion information on the reduced image, and then encodes the image with unchanged resolution based on the reduced resolution image marked with the motion information. This can reduce the occupation of hardware device resources due to the marking of motion information without losing motion information. Therefore, the system proposed by the implementation of the present application can improve the video encoding quality when hardware resources are compact.

[0034] The technical effects obtained by the above-mentioned second and third aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 A schematic diagram of the structure of a video encoding system provided in an embodiment of the present application;

[0037] Figure 2 A structural schematic diagram of a reading mode provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the structure of another video encoding system provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structure of another video encoding system provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of another video encoding system provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of a video encoding method provided in an embodiment of the present application;

[0042] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0044] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0045] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0046] The video encoding system provided in the present application can be used for an independent encoding device or an encoder in a video recording device such as a camera. Figure 1 A schematic diagram of the structure of a video encoding system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the above video encoding system includes:

[0047] The image processing unit 100 is used to output an image to be processed, where the image to be processed is generated based on a frame of image in a target video.

[0048] Exemplarily, the image processing unit 100 can process the video captured by the camera or sensor into frames of images to be processed for output. In this process, analog-to-digital signal conversion and image processing based on image processing, such as deblurring, color adjustment, sharpening, etc. are involved.

[0049] The online scaling unit 200 is connected to the image processing unit 100 for communication, and is used to reduce the resolution of the image to be processed.

[0050] The online information labeling unit 300 is in communication connection with the online scaling unit 200 . The online information labeling unit 300 is used to label the image to be processed after being processed by the online scaling unit 200 to generate a first labeled image. The first labeled image is labeled with motion information.

[0051] Exemplarily, the online scaling unit 200 is connected in communication with the image processing unit 100, and the online information labeling unit 300 is connected in communication with the online scaling unit 200. The online scaling unit 200 may be a scaler capable of reducing the resolution of the image to be processed, and the online information labeling unit 300 labels the motion information on the image processed by the online scaling unit 200 to form a first labeled image. The motion information includes position information, time information, etc. of each motion block in a frame of image.

[0052] It should be noted that by scaling the resolution of the image to be processed and then marking the motion information on the scaled image, the processing resources required for motion information marking can be reduced without losing the motion information contained in the image to be processed, thereby reducing the hardware burden of the video encoding device.

[0053] The image marking unit 400 is in communication connection with the image processing unit 100 . The image marking unit 400 is used to perform a marking operation on the image to be processed to generate a second marked image. The marking operation is used to mark a key area of ​​the image to be processed.

[0054] Exemplarily, the image marking unit 400 marks key areas in the image to be processed and generates a second marked image, for example, drawing lines, frames, OSD (On Screen Display) and other operations on some areas of the image involving people and moving targets to improve the quality of subsequent encoding.

[0055] The encoding unit 500 is respectively connected to the image marking unit 400 and the online information tagging unit 300 for communication. The encoding unit 500 is used to encode the second marked image based on the motion information in the first marked image.

[0056] Exemplarily, the encoding unit 500 reads the second marked image and the first marked image, and encodes the second marked image using motion information in the first marked image, thereby improving the encoding quality of the currently processed frame image and subsequently improving the quality of the final encoded video.

[0057] The video encoding system provided in the embodiment of the present application reduces the resolution of each frame of the target video, marks the motion information on the reduced image, and then encodes the image with unchanged resolution based on the reduced resolution image marked with the motion information. This can reduce the occupation of hardware device resources due to the marked motion information without losing the motion information. Therefore, the system proposed in the implementation of the present application can improve the video encoding quality when the hardware resources are compact.

[0058] In some examples, the online scaling unit 200 is connected to the image processing unit 100 by reading a hardware cache; the online scaling unit 200 is connected to the online information tagging unit 300 by reading a hardware cache.

[0059] For example, Figure 2 A structural diagram of a reading mode provided in an embodiment of the present application is shown as follows: Figure 2 As shown in the figure, data reading usually includes two ways, one is through the memory independent of the processor, such as DDR (Double Data Rate, double rate synchronous dynamic random access memory), and the other is through the hardware cache, that is, directly reading the processor's cache to obtain the required data. Compared with reading data from memory, directly reading the hardware cache is faster.

[0060] In the embodiment of the present application, the online scaling unit 200 is connected to the image processing unit 100 by reading the hardware cache, that is, the online scaling unit 200 directly reads the above-mentioned image to be processed from the hardware cache of the image processing unit 100, and the online information marking unit 300 is connected to the online scaling unit 200 by reading the hardware cache, that is, the online information marking unit 300 directly reads the image to be processed after the resolution is reduced by the online scaling unit 200 from the hardware cache of the online scaling unit 200 for processing. In this way, by directly reading data from the cache of each processing unit, the occupation of the memory bandwidth in the video processing device can be reduced, thereby further reducing the hardware burden.

[0061] Figure 3 A schematic diagram of the structure of another video encoding system provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, in some examples, the online information tagging unit 300 includes an online preset algorithm engine 310 and an online image tagging subunit 320;

[0062] The online preset algorithm engine 310 is respectively connected to the online scaling unit 200 and the online image marking subunit 320 for communication. The online preset algorithm engine 310 is used to mark motion information in the image to be processed after being processed by the online scaling unit 200 .

[0063] The online image marking subunit 320 is used to receive the image to be processed after being marked by a preset algorithm engine, and perform a marking operation on the image to be processed to generate a first marked image. The marking operation is used to mark key areas of the image to be processed.

[0064] For example, Figure 3The online preset algorithm engine 310 processes the image scaled by the online scaling unit 200 through a preset algorithm model. The preset algorithm model may be a GMM (Gaussian Mixture Model), which may support foreground and background modeling of the image. The online image marking subunit 320 is used to mark the key area of ​​the image processed by the online preset algorithm engine 310, such as drawing lines, drawing frames, OSD, etc. The key area may be the same as the key area marked by the above-mentioned image marking unit 400.

[0065] like Figure 3 As shown, in some examples, the image processing unit 100 includes: an image processing subunit 110, which is used to optimize a frame of image in the received target video to generate an image to be processed.

[0066] The online conversion subunit 120 is respectively connected to the image processing subunit 110, the image marking unit 400 and the online scaling unit 200 for communication. The online conversion subunit 120 is used to convert the image to be processed into images to be processed of at least two resolutions, and send the converted images to be processed of each resolution to the online scaling unit 200 and the image marking unit 400.

[0067] It should be noted that some cameras used in the field of security monitoring are often connected to different monitoring terminals. For example, one side is connected to the computer in the monitoring room, and the other side is connected to the mobile phone of the manager. The encoded image must be sent to both the computer and the mobile phone, and the two have different image resolution requirements when actually browsing. Therefore, different resolution image outputs are required for different terminals.

[0068] For example, Figure 3 As shown, each processing unit can adopt a single-thread processing mode. For example, for a frame of image in the target video, the online conversion subunit 120 first converts it into an image of a certain resolution. After the image is reduced in resolution by the online scaling unit 200, the motion information is annotated by the online information annotating unit 300. On the other hand, the image is marked by the image marking unit 400. Finally, the image annotated with motion information and the marked image are encoded by the encoding unit 500 to generate an image adapted to the resolution. Then, the situation of another resolution is processed in a similar manner.

[0069] It should be noted that the online conversion subunit 120 can be connected to the image processing subunit 110 and the online scaling unit 200 by reading the hardware cache. In this way, the online conversion subunit 120 can directly read data from the cache of the image processing subunit 110, and the online scaling unit 200 can also directly read data from the cache of the online conversion subunit 120 without passing through the memory, thereby saving memory bandwidth.

[0070] In some examples, such as Figure 2 and Figure 3 As shown, the above-mentioned video encoding system also includes an image signal processor 600, which is communicatively connected to the image processing unit 100. The image signal processor 600 is used to optimize each frame image in the received target video and send the optimized image to the image processing unit 100.

[0071] Exemplarily, the image signal processor 600 is an ISP (Image Signal Processor), which can convert the raw data captured by the camera sensor into a high-quality digital image through image conversion, AI technology, etc. In the embodiment of the present application, the ISP Figure 2 Each frame of the target video is sent to the image processing unit 100 via DDR. By introducing an ISP with artificial intelligence processing function before the image processing unit 100, the quality of subsequent video encoding can be improved.

[0072] In some examples, the number of image labeling units 400, online scaling units 200, and online information labeling units 300 is the same as that of at least two online scalers, and the image labeling units 400 and online scaling units 200 are each connected in one-to-one communication with at least two online scalers, and the online scaling unit 200 is connected in one-to-one communication with the online information labeling unit 300.

[0073] It should be noted that, in order to improve the processing speed so that the corresponding video can be quickly browsed on different terminals, the embodiment of the present application also provides a method: Figure 4 A structural diagram of another video encoding system provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, at least two online conversion sub-units 120 are each connected to a processing thread, and each processing thread includes an online scaling unit 200, an online information labeling unit 300, an image labeling unit 400, and an encoding unit 500. In this way, images of different resolutions can be output simultaneously through different online conversion sub-units 120, and images of different resolutions have a separate processing thread, thereby improving processing efficiency.

[0074] It is understandable that, in some cases where three or more terminals are connected, the processing method is similar to the above method and will not be repeated here.

[0075] In some examples, Figure 4 The online conversion subunit 120 includes at least two online scalers, and the at least two online scalers are respectively connected to the above-mentioned image processing subunit 110 and the above-mentioned online scaling unit 200 by reading the hardware cache. It should be noted that through at least two online scalers, the video encoding system provided in the embodiment of the present application can output at least two images with different resolutions, and the online scaler is connected to the image processing subunit 110 and the online scaling unit 200 by reading the hardware cache without memory processing. In this way, in the scenario where multiple resolution images need to be output, the system provided in the embodiment of the present application can save the bandwidth loss caused by time-division multiplexing in the related technology.

[0076] In some examples, Figure 4 The online scaling unit 200 in FIG. 2 is a proportional scaler.

[0077] It should be noted that the proportional scaler is to scale the original image at a fixed ratio. The processing logic of the scaler is relatively simple, so the amount of data processing can be reduced. In addition, the image processed by the online scaling unit 200 is handed over to the online information annotation unit 300 to annotate the motion information. By scaling in proportion, it is not easy to cause distortion of the motion information, thereby improving the accuracy of the annotated motion information, and then improving the final video encoding quality.

[0078] For example, Figure 5 A schematic diagram of the structure of another video encoding system provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, Figure 5 The main code stream has a resolution of 3840×2160, which can be used on a computer, and the auxiliary code stream has a resolution of 704×576, which can be used on a mobile phone. Figure 5 The solid line connection between the unit modules indicates that the two communicate with each other through memory (such as DDR), and the dotted line connection indicates that the two communicate with each other by reading hardware cache.

[0079] Figure 5The image processing subunit 110 includes an image processor. The image processor interacts with the ISP through memory. The ISP sends each frame of the target video to the image processor after its own optimization processing. The online conversion subunit 120 includes two online scalers, namely scaler0 and scaler1. Scaler0 reads the image to be processed from the image processor by reading the hardware cache and scales it to 3840×2160. At the same time, one of the scaled images is sent to the online scaling unit 200 by reading the hardware cache. The online scaling unit 200 is an equal-proportional scaler scaler1 / 4, which means that the length and width of the original image are scaled by 1 / 2, and the other is sent to the image marking unit 400 through memory. The image marking unit 400 includes an OSD processor for OSD operation. Scaler1 corresponds to the auxiliary code stream, and the processing method is similar to scaler0, which will not be repeated.

[0080] Considering that the ratio of the image resolution after scaling by scaler0 and scaler1 is close to 4, the same proportional scaler scaler1 / 4 is used for both the main and auxiliary code streams, and both are scaled by 1 / 4. Taking the auxiliary code stream as an example, the proportional scaler scaler1 / 4 reduces the received 704×576 resolution image to 176×144, and sends it to the online preset algorithm engine 310, i.e., GMM for VENC, i.e., Gaussian mixture model engine, by reading the hardware cache. After the online preset algorithm engine 310 performs motion information marking on the 176×144 resolution image, it is passed through the online image marking subunit 320, i.e., OSD for Mask. It should be noted that the processing methods of OSD for Mask and OSD processor are the same. The 176×144 resolution image is sent to the encoding unit 500, namely VENC (Video encoder) through OSD for Mask. At the same time, the 704×576 resolution image is also sent to VENC through OSD processor. After VENC encoding, the encoded image is output. This is done for each frame of the target video, thereby completing the encoding of the video. The main code stream is processed in the same way and will not be described in detail.

[0081] The embodiment of the present application further provides a video encoding method, which is used in the video encoding system as described in any one of the first aspects. The execution subject of the method may be a main processor of a camera. Figure 6 A schematic diagram of a video encoding method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the above video encoding method includes:

[0082] S110 , in response to an instruction of the target terminal, controlling the image processing unit 100 to convert a received image into an image to be processed, where the received image is each frame image in the target video.

[0083] For example, in the field of security, the target terminal may be a computer in a monitoring room or a mobile phone of a security guard. When the target terminal issues an instruction to call a surveillance video, the image processing unit 100 in the surveillance camera may convert the recorded video frame by frame into images to be processed.

[0084] It should be noted that the above-mentioned instruction of the target terminal may include information such as the type of the target terminal, the required clarity (resolution), etc.

[0085] S120 , controlling the online scaling unit 200 to scale the image to be processed, and sending the scaled image to be processed to the online information tagging unit 300 .

[0086] Exemplarily, the online scaling unit 200 scales the image to be processed, and the resolution after scaling may be based on the information included in the target terminal instruction.

[0087] S130 , controlling the online information labeling unit 300 to label the scaled image to be processed with motion information to generate a first labeled image.

[0088] S140 , controlling the image marking unit 400 to mark the key area of ​​the image to be processed to generate a second marked image.

[0089] Exemplarily, the key areas may be some key locations in the surveillance image, such as a doorway, or people, moving objects, etc. in the image.

[0090] S150 , controlling the encoding unit 500 to encode the second marked image based on the motion information in the first marked image to obtain an encoded image.

[0091] Exemplarily, after acquiring the first marked image and the second marked image, the encoding unit 500 encodes the second marked image based on the motion information in the first marked image.

[0092] S160: Send the encoded image to the target terminal according to the encoding order.

[0093] Exemplarily, each frame of the image is processed according to the above steps S110 to S150 and sent to the target terminal in sequence to complete the encoding of the video.

[0094] The embodiment of the present application also provides an electronic device, a memory for storing a computer program;

[0095] A processor is used to execute the computer program to implement the steps of the method described in the second aspect.

[0096] For example, Figure 7 As shown, the electronic device includes a processor 710, at least one communication bus 720, a user interface 730, at least one external communication interface 740 and a memory 750. The communication bus 720 is configured to realize the connection and communication between these components. The user interface 730 may include a display screen, and the external communication interface 740 may include a standard wired interface and a wireless interface. The memory 750 stores a computer program. The processor 710 is used to execute the computer program stored in the memory 750.

[0097] The sequence of serial numbers or introduction of the embodiments of the present application is for description only and does not represent the superiority or inferiority of the embodiments.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0100] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0101] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0102] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the scene data of the current frame in the three-dimensional virtual scene, the device information of the client, and the scene interaction information involved in the embodiments of this application are all obtained with full authorization.

[0103] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A video encoding system, characterized in that: include: An image processing unit, the image processing unit is used to output an image to be processed, wherein the image to be processed is generated based on a frame of image in a target video; An online scaling unit, connected to the image processing unit for communication, and configured to reduce the resolution of the image to be processed; an online information labeling unit, which is in communication with the online scaling unit, and is used to label the image to be processed after being processed by the online scaling unit to generate a first labeled image, wherein the first labeled image is labeled with motion information; an image marking unit, which is in communication with the image processing unit, and is used to perform a marking operation on the image to be processed to generate a second marked image, wherein the marking operation is used to mark a key area of ​​the image to be processed; The encoding unit is respectively connected to the image marking unit and the online information tagging unit for communication, and the encoding unit is used to encode the second marked image based on the motion information in the first marked image.

2. The video encoding system according to claim 1, characterized in that The image processing unit comprises: An image processing subunit, the image processing subunit being used to optimize a frame of image in the received target video to generate the image to be processed; An online conversion subunit, wherein the online conversion subunit is respectively communicatively connected with the image processing subunit, the image marking unit and the online scaling unit, and the online conversion subunit is used to convert the image to be processed into images to be processed of at least two resolutions, and send the converted images to be processed of each resolution to the online scaling unit and the image marking unit.

3. The video encoding system according to claim 2, characterized in that The number of the image marking unit, the online scaling unit, and the online information labeling unit is the same as the number of the at least two online conversion sub-units, and the image marking unit and the online scaling unit are both connected to the at least two online conversion sub-units in a one-to-one communication manner, and the online scaling unit is connected to the online information labeling unit in a one-to-one communication manner.

4. The video encoding system according to claim 3, characterized in that The online conversion subunit includes at least two online scalers, and the at least two online scalers are respectively connected to the image processing subunit and the online scaling unit by reading hardware cache.

5. The video encoding system according to claim 3, characterized in that The online scaling unit is a proportional scaler.

6. The video encoding system according to claim 1, characterized in that Also includes: An image signal processor is communicatively connected to the image processing unit, and is used for optimizing each frame of the received target video and sending the optimized image to the image processing unit.

7. The video encoding system according to claim 1, characterized in that The online information marking unit includes an online preset algorithm engine and an online image marking subunit; The online preset algorithm engine is respectively connected to the online scaling unit and the online image marking subunit for communication, and the online preset algorithm engine is used to mark motion information in the image to be processed after being processed by the online scaling unit; The online image marking subunit is used to receive the image to be processed after being marked by the preset algorithm engine, and perform a marking operation on the image to be processed to generate the first marked image, wherein the marking operation is used to mark the key area of ​​the image to be processed.

8. The video encoding system according to any one of claims 1 to 7, characterized in that: The online scaling unit is connected to the image processing unit by reading a hardware cache; The online scaling unit is connected to the online information marking unit by reading a hardware cache.

9. A video encoding method, characterized in that: Used in a video encoding system according to any one of claims 1 to 8, the video encoding method comprising: In response to an instruction from the target terminal, the image processing unit is controlled to convert the received image into an image to be processed, wherein the received image is each frame image in the target video; Controlling the online scaling unit to scale the image to be processed, and sending the scaled image to be processed to the online information labeling unit; Controlling the online information labeling unit to label the scaled image to be processed with motion information to generate a first labeled image; Controlling the image marking unit to mark the key area of ​​the image to be processed to generate a second marked image; Controlling the encoding unit to encode the second marked image based on the motion information in the first marked image to obtain an encoded image; The encoded images are sent to the target terminal in an encoding order.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the method according to claim 9.

Images