Video coding by object identification and feature extraction.

Abstract

A transmitter and a receiver are provided for processing machine-directed video. The transmitter is configured to (1) identify one or more objects in a video and extract one or more features associated with the one or more objects, (2) process each frame of the video based on the one or more objects, and (3) encode the processed video. The receiver is configured to (i) receive and decode the encoded video, and (ii) generate a video based on the extracted features and the decoded video.

Description

[Technical field]

[0001] This application relates to video coding, in particular to video coding based on object identification and feature extraction, and also to image synthesis based on said video coding. [Background technology]

[0002] Video compression has been used to transmit video data. The higher the compression ratio, the less resources are required for transmission, but the more the quality of the video is lost. For videos that are viewed by people, the quality loss affects the aesthetics of the video, and accordingly worsens the user experience. However, for images that are identified by machines (e.g., autonomous vehicles), the context of the image is more important than the aesthetics of the video. The latest trends in Video Coding for Machines (VCM) can be found in ISO / IEC JTC 1 / SC 29 / WG 2 N18, "Use Cases and Requirements for Video Coding for Machines." In order to reduce the transmission time and consumption of transmission resources for machine-oriented video data, improved systems and methods for effectively encoding and decoding machine-oriented video data are desirable. Summary of the Invention

[0003] The present disclosure provides an apparatus and method for processing video data for a machine. In some embodiments, the machine may include an autonomous vehicle, a robot, an aircraft, and / or other suitable device or computing system capable of processing and analyzing video data, e.g., using artificial intelligence. More specifically, the present disclosure provides (i) a transmitter configured to encode or compress video data based on identified objects in the video and / or features of the video, and (ii) a receiver configured to decode or decompress the video data encoded or compressed by the transmitter.

[0004] When encoding the video, the transmitter may (i) identify one or more objects in the video (e.g., traffic signs, road signs, logos, tables, other suitable regions / fields providing textual and / or numerical information, etc.), (ii) extract features associated with the identified objects (e.g., text, numbers, and their corresponding colors, fonts, sizes, positions, etc.), (iii) monitor and / or track the identified objects to determine or predict their movement direction and / or trajectory, (iv) process images corresponding to the identified objects in each frame of the video (e.g., filling the entire area occupied by the identified objects with a representative color, such that the resolution of the area is significantly reduced), (v) encode (or compress) the video with the processed images, and (vi) transmit the encoded (or compressed) video and the extracted features over a network (e.g., in a bitstream). An embodiment of the transmitter is described in detail with reference to FIG. 3.

[0005] The present disclosure also provides a receiver configured to decode the encoded video. In some embodiments, the receiver can (a) receive the encoded video over a network, (b) decode the encoded video based on the identified objects and their corresponding characteristics, and (c) generate a decoded video having the identified objects. Receiver This embodiment will be described in detail with reference to FIG.

[0006] One aspect of the present disclosure provides a method for processing a video having objects. The method may, for example, include: (1) identifying one or more objects in the video; (2) extracting features associated with the identified objects; (3) determining the location, movement direction, and / or trajectory of the identified objects; (4) processing images corresponding to the identified objects in each frame of the video; (5) generating a descriptor corresponding to the extracted features; (6) compressing the generated descriptor; (7) encoding the video having the processed images (e.g., encoding the processed images and the remaining portion of the video, respectively); and (8) transmitting the encoded video and the compressed descriptor (e.g., by a multiplexed bitstream). In some embodiments, the method may further include: (9) receiving the encoded video and the compressed descriptor over a network; (10) decompressing the compressed descriptor; and (11) decoding the encoded video based on the decompressed descriptor. An embodiment of the method is described in detail with reference to FIG. 5 and FIG. 9. In some embodiments, the objects may include signs, advertisements, directional / traffic signs, and the like.

[0007] In some embodiments, the methods may be realized by a tangible, non-transitory computer readable medium having processor instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform one or more aspects / features of the methods described herein. [Brief description of the drawings]

[0008] In order to more clearly describe the technical solutions of the embodiments of the present disclosure, the drawings are briefly described below. The drawings only show some aspects or embodiments of the present disclosure, and those skilled in the art can obtain other drawings from these drawings without creative efforts. [Figure 1]1 is a schematic diagram illustrating a wireless communication system according to one or more embodiments of the present disclosure. In some embodiments, the present technology can also be implemented in a wired communication system. [Diagram 2] FIG. 2 is a schematic diagram illustrating a system according to one or more embodiments of the present disclosure. [Diagram 3] FIG. 3 is a schematic diagram illustrating a transmitter according to one or more embodiments of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating a receiver according to one or more embodiments of the present disclosure. [Diagram 5] FIG. 5 is a schematic diagram illustrating image processing of an object according to one or more embodiments of the present disclosure. [Figure 6A] FIG. 6A is an example of an image processed by the method of the present disclosure. [Figure 6B] FIG. 6B is an example of an image processed by the method of the present disclosure. [Figure 6C] FIG. 6C is an example of an image processed by the method of the present disclosure. [Figure 6D] FIG. 6D is an example of an image processed by the method of the present disclosure. [Figure 7A] FIG. 7A is an example of an image processed by the method of the present disclosure. [Figure 7B] FIG. 7B is an example of an image processed by the method of the present disclosure. [Figure 8A] FIG. 8A is an example of an image processed by the method of the present disclosure. [Figure 8B] FIG. 8B is an example of an image processed by the method of the present disclosure. [Figure 9] FIG. 9 is a flowchart illustrating a method according to one or more embodiments of the present disclosure. [Figure 10] FIG. 10 is a block diagram illustrating a terminal device according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Communication environment FIG 1 shows a system 100 for implementing the method of the present disclosure. As shown in FIG 1, the system 100 includes a network device 101. 101 Examples of the include a base transceiver station (BTS), a Node B (NB), an evolved Node B (eNB or eNodeB), a next generation Node B (gNB or gNode B), a wireless fidelity (Wi-Fi) access point (AP), etc. In some embodiments, the network device 101 may include relay stations, access points, in-vehicle devices, wearable devices, etc. 101may include wireless access devices used in communication networks, such as global system for mobile communication (GSM) networks, code division multiple access (CDMA) networks, wideband CDMA (WCDMA) networks, long term evolution (LTE) networks, Cloud Radio Access Networks (CRAN), Institute of Electrical and Electronics Engineers (IEEE) 802.11-based networks (e.g., Wi-Fi networks), internet of things (IoT) networks, device to device (D2D) networks, next generation networks (e.g., 5G networks), future evolution public land mobile networks (PLMN), vehicle to vehicle (V2V) communications, vehicle-to-infrastructure (V2I) communications, etc. 5G systems or networks may be referred to as new radio (NR) systems or networks.

[0010] As shown in FIG. 1, the system 100 further includes a terminal device 103. The terminal device 103 is a network device. 101The terminal device 103 may be an end user device configured to communicate with the network device 101. The terminal device 103 may be configured to connect wirelessly (e.g., via radio channel 105) to the network device 101 according to one or more corresponding communication protocols / standards. The terminal device 103 may be mobile, fixed, wired, tethered, or untethered. The terminal device 103 may be a user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user device. Examples of terminal devices 103 include modems, cellular phones, smartphones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, wearable devices, IoT devices, terminal devices in future 5G networks, terminal devices in future evolved PLMNs, etc.

[0011] 1 illustrates only one network device 101 and one terminal device 103 in the wireless communication system 100. However, as can be appreciated, in some examples, the wireless communication system 100 may include additional devices / other devices, such as additional instances of the network device 101 and / or the terminal device 103, a network controller, mobility management entities / devices, etc.

[0012] In some embodiments, network device 101 can operate as a transmitter as described herein. Or, in some embodiments, network device 101 can operate as a receiver as described herein. Similarly, terminal device 103 can operate as a transmitter as described herein. Or, in some embodiments, terminal device 103 can operate as a receiver as described herein.

[0013] FIG. 2 is a schematic diagram illustrating a system 200 according to one or more embodiments of the present disclosure. The system 200 includes a transmitter 201 and a receiver 203. The transmitter 201 is configured to transmit encoded video data to the receiver 203 over a network 205. The transmitter 201 includes a processor 2011, a memory 2013, and an encoder 2015. In some embodiments, the transmitter 201 can be implemented as a chip (e.g., a system-on-chip, SoC). The processor 2011 is configured to implement the functions of the transmitter 201 and its components. The memory 2013 is configured to store data, instructions, and / or information related to the transmitter 201. The encoder 2015 is configured to process and encode a video having one or more objects. In some embodiments, the encoder 2015 may (1) identify one or more objects in a video, (2) extract one or more features associated with the one or more objects, (3) process each frame of the video based on the one or more objects, (4) encode the processed video, and (5) transmit the encoded video and the extracted features.

[0014] In some embodiments, the Encoder 2015 may be used to process video data for a machine (e.g., a vehicle, an aircraft, a ship, a robot, or other suitable device or computing system capable of processing and analyzing video data using artificial intelligence, etc.). The Encoder 2015 may first identify one or more objects in the video. Examples of objects may include, for example, traffic signs, road signs, company / business signs (e.g., signs that say "Coca-Cola", "McDonald's", etc.), pictograms, logos, other suitable areas / fields that provide textual and / or numerical information, etc. In some embodiments, the objects may be defined by a system operator (e.g., having a particular shape, a particular color, particular textual characteristics, etc.).

[0015] Once an object is identified, the encoder 2015 can extract one or more features from the identified object. Examples of extracted features include text, numbers, and their corresponding color, font, size, position, etc., associated with the identified object. For example, a traffic sign in a video can be identified as an object, and the information "Maximum speed: 100km / h" on the traffic sign can be an extracted feature. By isolating the information carried on the traffic sign, the video containing the traffic sign can be compressed at a higher rate (corresponding to a smaller size of data for transmission) without worrying about the information becoming unidentifiable due to compression.

[0016] The encoder 2015 may also process the video by removing images associated with objects in each frame of the video. In some embodiments, these images associated with objects may be replaced with a traffic light color (e.g., a representative color that is the same or similar to the surrounding image) or a background image (e.g., a default background of a traffic sign). In some embodiments, these images may be blanked (later generated by a decoder). The processed video (i.e., the video with the objects removed, replaced, or edited) may then be encoded (e.g., as a bitstream) for transmission.

[0017] In some embodiments, the encoder 2015 may be configured to track or monitor the identified object, allowing the position, movement direction, and / or trajectory of the object in the incoming frame to be determined or predicted. For example, the encoder 2015 may set several positions (e.g., pixels) surrounding the object as "checkpoints" to track or monitor possible position changes of the object. Such setting allows the encoder 2015 to effectively identify and manage the object without losing track of the object. In some embodiments, information regarding the object's boundary may be tracked and / or updated on a frame-by-frame basis.

[0018] The encoded video and the extracted features may then be transmitted over a network 205. In some embodiments, the encoded video and the extracted features may be transmitted in two bitstreams. In some embodiments, the encoded video and the extracted features may be transmitted in the same bitstream.

[0019] As shown in FIG. 2, the receiver 203 can receive the encoded video and then "reconstruct" the encoded video by encoding the video and adding the extracted features to the video. For example, in some embodiments, objects can be modified and added based on corresponding descriptors (e.g., gaze direction, object size / shape, etc.). The receiver 203 includes a processor 2031, a memory 2033, and a decoder 2035. In some embodiments, the receiver 203 can be implemented as a chip (e.g., SoC). The processor 2031: Receiver 203 and its components. The memory 2033 is configured to store data, instructions, and / or information related to the receiver 203. The decoder 2035 is configured to decode the encoded video and restore removed / replaced objects in the video. In some embodiments, the decoder 2035 can perform functions in a "reverse" manner to the encoder 2015 to restore the video. In some embodiments, the decoder 2035 can further process the video for better image quality or generate the video based on user preferences.

[0020] In some embodiments, both the transmitter 201 and the receiver 203 may further comprise an object database configured to store reference object information (e.g., type of object, sample object for comparison, etc.) used to identify one or more objects. In some embodiments, the information stored in the object database may be trained by a machine learning process to improve the accuracy of identifying objects.

[0021] In some embodiments, the extracted features can be described with a descriptor. The descriptor indicates the extracted feature's textual information (e.g., text table, road name, etc.), numerical information (e.g., indicated numbers), location information (e.g., relative position of the object, direction of movement, etc.), contextual information (that the object is adjacent to a building or road), and / or graphic information (e.g., color, size, shape, etc.). The descriptor can, for example, be stored in an object database, fully or partially. For example, the descriptor (e.g., traffic signs) can be stored in the object database and only the parameters of the descriptor (e.g., the size, location, and parameters of the affine transformation that defines their appearance) are transmitted.

[0022] In some embodiments, the encoding, decoding, compression, and decompression processes described herein may include coding processes according to advanced video coding (AVC), high efficiency video coding (HEVC), versatile video coding (VVC), Alliance for Open Media Video 1 (AV1), or any other suitable method, protocol, or standard.

[0023] 3 is a schematic diagram illustrating a transmitter 300 according to one or more embodiments of the present disclosure. The transmitter 300 includes an object database 310, an object identification component 311, a video processing component 312, a compression component 313, a video encoder 314, and a bitstream multiplexer (or transmission component) 315. The above components may be controlled or managed by a processor of the transmitter 300.

[0024] When an input video 31 enters the sender 300, it may travel to an object identification component 311 and a video processing component 312. In some embodiments, the input video 31 may reach the object identification component 311 first and then the video processing component 312.

[0025] The object identification component 311 is configured to identify one or more objects in the video. As shown in FIG. 3, the object identification component 311 is coupled to an object database 310. The object database 310 stores reference object information (e.g., type of object, sample object for comparison, etc.) used to identify one or more objects in the video. In some embodiments, the information stored in the object database 310 can be trained by a machine learning process to increase the accuracy of identifying the objects. The object identification component 311 can send queries and receive query responses 36 from the object database 310. The query responses 36 can facilitate the object identification component 311 to identify and / or determine one or more objects in the input video 31.

[0026] Once one or more objects are identified, one or more features associated with the one or more objects can be extracted. Examples of extracted features include text, numbers, and their corresponding color, font, size, location, etc. associated with the identified objects. One or more descriptors 34 can be generated based on the extracted features. The descriptors 34 indicate the features of the one or more objects (e.g., what the features are, where they are located, etc.). The descriptors 34 are sent to the video processing component 312 and the compression component 313 for further processing.

[0027] After receiving the descriptor 34, the compression component 313 compresses the descriptor 34 to generate a compressed descriptor 35. In some embodiments, the compression ratio of the compression can be determined based on the content of the descriptor 34. The compressed descriptor 35 is then sent to the bitstream multiplexer 315 for further processing.

[0028] After receiving the descriptor 34, the video processing component 312 processes the input video by removing the identified objects in the video (e.g., based on information provided by the descriptor 34). The video processing component 312 then generates a processed video 32 (with the identified objects removed). In some embodiments, the removed objects can be replaced with blanks, background colors, background images, or suitable items that have a lower image resolution than the removed objects. Examples of blanks, background colors, and background images are described in more detail with reference to FIG. 5. The processed video 32 is then sent to the video encoder 314 for further processing.

[0029] The video encoder 314 then encodes the processed video 32 using a video coding scheme (e.g., AVC, HEVC, VVC, AV1, or any other suitable method, protocol, or standard). The video encoder 314 then generates an encoded video 33, which is sent to a bitstream multiplexer 315 for further processing.

[0030] After receiving the encoded video 33 and the compressed descriptor 35, the bitstream multiplexer 315 may generate a multiplexed bitstream 37 for transmission. In some embodiments, the multiplexed bitstream 37 may include two bitstreams (i.e., one for the encoded video 33 and one for the compressed descriptor 35). In some embodiments, the multiplexed bitstream 37 may be a single bitstream. In some embodiments, the transmitter 300 may be implemented without the multiplexed bitstream 37.

[0031] 4 is a schematic diagram illustrating a receiver 400 according to one or more embodiments of the present disclosure. The receiver 400 includes a bitstream demultiplexer (or receiving component) 415, an object description decoder 413, an object reconstruction component 411, an object database 410, a video decoder 414, and a video merging component 412.

[0032] The bitstream demultiplexer 415 receives and multiplexes the multiplexed compressed bitstream 40. Thus, the bitstream demultiplexer 415 can generate compressed descriptors 41 and encoded videos 42. The encoded videos 42 are sent to the video decoder 414. The video decoder 414 then decodes the encoded videos 42 to generate decoded videos 44 (with objects removed). The decoded videos 44 are sent to the video synthesis component 412 for further processing.

[0033] The compressed descriptor 41 is sent to an object description decoder 413. The object description decoder 413 may decode the compressed descriptor 41 and generate a descriptor 43. The descriptor 43 indicates the extracted feature or features that correspond to one or more objects. The descriptor 43 is sent to the object reconstruction component 411 for further processing.

[0034] The object reconstruction component 411 is coupled to the object database 410. The object database 410 stores reference object information (e.g., type of object, sample object for comparison, etc.) used to identify the corresponding one or more objects based on the descriptors 43. In some embodiments, the information stored in the object database 410 can be trained by a machine learning process to improve the accuracy of identifying the objects. The object reconstruction component 411 can send queries and receive query responses 45 from the object database 410. The query responses 45 can facilitate the object reconstruction component 411 to identify the one or more objects indicated by the descriptors 43. Thus, the object reconstruction component 411 can generate a reconstructed object 46. The reconstructed object 46 can be sent to the video integration component 412 for further processing. In some embodiments, the reconstructed object 46 can also be sent and utilized for reference or machine vision / machine learning studies.

[0035] After receiving the reconstructed object 46 and the decoded video 44, the video merging component 412 merges the reconstructed object 46 with the decoded video 44 to generate a decoded video with the object 47. The decoded video with the object 47 has a resolution suitable for people (and machines) to identify the object in the video.

[0036] FIG. 5 is a schematic diagram illustrating image processing of an object 501 according to one or more embodiments of the present disclosure. As shown in FIG. 5, the object 501 is in an image 500 that includes a number of grids 50. Once the object 501 is identified or recognized, the image 500 can be processed to remove the object 501 by performing an "over-the-hole" process. For example, in the "over-the-hole" process, the grids occupied by the object 501 can be removed. These grids can be replaced with blanks 503, background images 505, or values ​​interpolated by the neighboring grids 51, 52. This arrangement allows the object 501 to be removed from the image 500, and an object-removed image 507 can be generated and processed.

[0037] FIG. 6A is an example of an image processed by the method in the present disclosure. FIG. 6B is an example of an image processed by the method in the present disclosure. FIG. 6C is an example of an image processed by the method in the present disclosure. FIG. 6A shows an original image 600A including a traffic sign 601 (i.e., an object). FIG. 6B shows a processed image 600B in which the traffic sign 601 has been removed and replaced with a background color. The processed image 600B can be transmitted at a high image compression rate without worrying about losing information carried by the traffic sign 601. In some embodiments, the traffic sign 601 can be "partially removed" by reducing its resolution, as in the processed image 600C shown in FIG. 6C. After transmission, the processed image 600B or 600C can be restored or "inpainted" to a restored image 600D shown in FIG. 6D.

[0038] Fig. 7A is an example of an image processed by the method in the present disclosure. Fig. 7B is an example of an image processed by the method in the present disclosure. In the embodiment shown in Fig. 7A and Fig. 7B, an original image 700A includes two objects, a traffic sign 701 and a lane marking 702. As shown in Fig. 7B, both of the objects can be removed as shown in the processed image 700B.

[0039] In some embodiments, there may be more than two objects in an image. In FIG. 8A, original image 800A includes multiple potential objects. The system and method allow an operator to determine criteria for identifying objects and to determine whether to process images associated with the objects. For example, as shown in FIG. 8B, six objects 801-806 are identified and processed. However, in other embodiments, the operator may determine to process only a portion of the identified objects (e.g., to process only objects 801-804 and not objects 805, 806). The system allows an operator to customize the object identification process (e.g., what types of objects to identify) and to determine whether to process specific types of objects.

[0040] 9 is a flow chart illustrating a method 900 according to one or more embodiments of the present disclosure. The method 900 may be implemented by a system (such as system 200), a transmitter (such as transmitter 201 or 300), and / or a receiver (such as receiver 203 or 400) to process video.

[0041] At block 901, method 900 begins with identifying one or more objects in a video. Examples of objects may include, for example, traffic signs, road signs, other suitable regions / fields that provide textual and / or numerical information, etc. In some embodiments, the objects may be defined by a system operator (e.g., having a particular shape, a particular color, particular textual characteristics, etc.).

[0042] At block 903, the method 900 continues with extracting features associated with the identified object. Examples of extracted features include text, numbers, and their corresponding colors, fonts, sizes, positions, etc. associated with the identified object. For example, a traffic sign in a video may be identified as an object, and the information "Maximum speed: 100km / h" on the traffic sign may be an extracted feature.

[0043] At block 905, the method 900 continues to process images corresponding to the identified objects in each frame of the video. In some embodiments, the images may be processed by removing the objects in the images (see, e.g., FIG. 5).

[0044] At block 907, the method 900 continues by generating descriptors corresponding to the extracted features. In some embodiments, the descriptors indicate the extracted features' textual information (e.g., text table, road name, etc.), numerical information (e.g., numbers shown), location information (e.g., relative position of the object, direction of movement, etc.), contextual information (that the object is adjacent to a building or road), and / or graphical information (e.g., color, size, shape, etc.). The descriptors can be stored in an object database.

[0045] At block 909, the method 900 continues with encoding the generated descriptors and the video with the processed images.

[0046] At block 911, method 900 continues with transmitting the encoded video and the descriptor. In some embodiments, the descriptor may be compressed with a first scheme, while the video with processed images may be encoded with a second scheme. In some embodiments, the first scheme and the second scheme may be the same. In some embodiments, the first scheme and the second scheme may be different. In some embodiments, the encoding, decoding, compression, and decompression processes described herein may include coding processes according to AVC, HEVC, VVC, AV1, or any other suitable method, protocol, or standard.

[0047] In some embodiments, the encoded video and the descriptor may be multiplexed and transmitted in a single bitstream or two bitstreams. In some embodiments, the method 900 may further include receiving the encoded video and the compressed descriptor over a network, decompressing the compressed descriptor, and decoding the encoded video based on the decompressed descriptor.

[0048] 10 is a block diagram illustrating a terminal device 1000 (e.g., an example of the terminal device 103 of FIG. 1) according to one or more embodiments of the present disclosure. As shown in FIG. 10, the terminal device 1000 includes a processing unit 1010 and a memory 1020. The processing unit 1010 may be configured to execute instructions corresponding to the terminal device 1000.

[0049] As can be understood, the processor in the embodiment of the present technology can be an integrated circuit chip having signal processing capabilities. In the implementation process, the steps in the above method can be realized by an integrated logic circuit in the form of hardware in the processor or instructions in the form of software. The processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The processor can realize or execute the method, steps and logic block diagrams disclosed in the embodiment of the present technology. The general-purpose processor can be a microprocessor or any ordinary processor, etc. The steps of the method disclosed in the embodiment of the present technology can be directly performed or completed by a hardware decoding processor, or can be performed or completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well known in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory. The processor reads the information in the memory and completes the steps of the above method in conjunction with the processor hardware.

[0050] As can be understood, the memory in the embodiment of the present technology can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) that functions as an external high-speed cache. By way of illustrative, but non-limiting examples, various RAMs are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchronous link DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DRRAM). It should be noted that memory in the systems and methods described herein may include, but is not limited to, these and any other suitable types of memory.

[0051] The above detailed description of examples of the disclosed technology is not intended to be exhaustive or to limit the disclosed technology to the precise form disclosed above. Although specific examples of the disclosed technology are described above for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the described technology. For example, while processes or blocks are shown in a certain order, alternative embodiments may perform routines having steps or employ systems having blocks in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and / or modified to provide alternative embodiments or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, while processes or blocks may be shown to be performed in series, these processes or blocks may be performed or implemented in parallel, or may be performed at different times. Furthermore, specific numerical values ​​described herein are merely examples, and alternative embodiments may employ different values ​​or ranges.

[0052] In the detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently described technology. In other embodiments, the technology introduced herein may be implemented without these specific details. In other instances, well-known features, such as specific functions or routines, are not described in detail to avoid unnecessarily obscuring the present disclosure. References herein to "embodiments / examples," "one embodiment / example," and the like, mean that the particular features, structures, materials, or characteristics described are included in at least one embodiment of the described technology. Thus, appearances of such phrases herein do not necessarily all refer to the same embodiment / example. On the other hand, such phrases are not necessarily mutually exclusive. Moreover, particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments / examples. As can be appreciated, the various embodiments shown in the figures are merely illustrative and are not necessarily drawn to scale.

[0053] For purposes of clarity, some details describing structures or processes are not described herein, which are well known and typically associated with communication systems and subsystems, but may unnecessarily obscure some important aspects of the disclosed technology. Furthermore, the following disclosure illustrates some embodiments of different aspects of the disclosure, but some other embodiments may have different configurations or different components than those described in this section. Thus, the disclosed technology may have other embodiments that have additional elements or that do not have some of the elements described.

[0054] Many embodiments or aspects of the technology described herein may take the form of computer-executable or processor-executable instructions, including routines executed by a programmable computer or processor. Those skilled in the art will appreciate that the described technology may be implemented in computer or processor systems other than those shown and described. The technology described herein may be implemented in a special-purpose computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the described computer-executable instructions. Thus, the term "processor" as used generally herein refers to any data processor. Information manipulated by a processor may be presented in any suitable display medium. Instructions for performing computer-executable or processor-executable tasks may be stored in any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. Instructions may be included in any suitable memory device, including, for example, a flash drive and / or other suitable medium.

[0055] The terms "coupled" and "connected" and their derivatives may be used herein to indicate a structural relationship between components. As can be understood, these terms are not intended to be synonyms of each other. Rather, in certain embodiments, "connected" may be used to indicate that two or more elements are in direct contact with each other. Unless otherwise clearly explained by the context, the term "coupled" may be used to indicate that two or more elements are in direct or indirect contact with each other (with other intermediate elements between the elements), or that two or more elements cooperate or interact with each other (e.g., in a causal relationship, for signal transmission / reception, or for function calls), or that they cooperate and interact with each other. The term "and / or" in this specification is merely intended to describe the relationship between related objects, and indicates that three types of relationships exist. For example, A and / or B indicates three situations: only A exists, A and B exist simultaneously, and only B exists.

[0056] These and other changes can be made to the disclosed technology in light of the above detailed description. Although the detailed description has set forth certain examples and the best contemplated modes of the disclosed technology, the disclosed technology can be implemented in many ways, no matter how detailed the above description is herein. The details of the system may vary considerably in its specific embodiment, but still be covered by the technology disclosed herein. As mentioned above, a specific term used in describing a particular feature or aspect of the disclosed technology should not be taken to imply that the term is redefined herein to be limited to any particular characteristic, feature, or aspect of the disclosed technology associated with the term. Thus, the present invention is not limited except as set forth in the appended claims. In general, the terms used in the claims should not be interpreted as limiting the disclosed technology to the specific examples disclosed herein, unless such terms are expressly defined in the above detailed description.

[0057] Those skilled in the art may recognize that, in conjunction with the examples described in the embodiments disclosed herein, the units and algorithm steps may be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether a function is performed by hardware or software is determined by the specific application of the technical solution, design constraints, etc. Those skilled in the art may realize the described functions using different methods for each specific application, but these realizations should not be considered as going beyond the scope of this application.

Claims

1. 1. A transmitter system for processing video, comprising: an object identification component configured to identify one or more objects in the video and extract one or more features associated with the one or more objects; a video processing component configured to process each frame of the video by removing the one or more objects; a video encoding component configured to encode the processed video; a transmission component configured to transmit the encoded video and the extracted one or more features.

13. A transmitter system for processing video, comprising:

2. and wherein the transmitter system for processing the video further comprises an object database configured to store reference object information used to identify the one or more objects. A transmitter system for processing video according to claim 1.

3. The one or more objects include at least one of the following: a traffic sign, a road sign, an information table, a company / product / organization logo, or an area or field providing textual and / or numerical information; A transmitter system for processing video according to claim 1.

4. the one or more extracted features include at least one of text, numbers, color, font, size, or position associated with the one or more objects; A transmitter system for processing video according to claim 1.

5. the video processing component is configured to replace the one or more objects in each frame of the video with a blank; or the video processing component is configured to replace the one or more objects in each frame of the video with a background color; or the video processing component is configured to replace the one or more objects in each frame of the video with a background image. A transmitter system for processing video according to claim 1.

6. the background image is determined based on images adjacent the one or more objects; or the background image is determined based on an image surrounding the one or more objects; A transmitter system for processing video according to claim 5.

7. the object identification component is further configured to monitor the one or more identified objects to determine a direction of movement of the one or more identified objects. A transmitter system for processing video according to claim 1.

8. 1. A receiver system for processing video, comprising: a receiving component, a video decoding component, an object reconstruction component, and a video synthesis component; the receiving component is configured to receive an encoded video and one or more extracted features, wherein one or more objects of the encoded video have been removed, and the extracted one or more features are associated with the one or more objects; the video decode component configured to decode the encoded video; the object reconstruction component is configured to generate an image based on the extracted one or more features; the video merging component is configured to merge the image and the decoded video based on the extracted one or more features.

13. A receiver system for processing video, comprising:

9. and wherein the receiver system for processing the video further comprises an object database configured to store reference object information used to identify the one or more objects. Receiver system for processing video according to claim 8.

10. the extracted one or more features include at least one of text, numbers, color, font, size, or position associated with the one or more objects; Receiver system for processing video according to claim 8.

11. the object reconstruction component is configured to generate the image by applying the extracted one or more features to the one or more objects. Receiver system for processing video according to claim 8.

12. 1. A method for processing a video, comprising: identifying one or more objects in the video; extracting features associated with the identified one or more objects; processing images corresponding to the identified one or more objects in each frame of the video; generating a descriptor corresponding to the extracted features; compressing the generated descriptor; encoding the video having the processed images; transmitting the encoded video and the encoded descriptor.

13. A method for processing video, comprising:

13. The method of claim 12, further comprising: processing images corresponding to the one or more identified objects in each frame of the video, the processing comprising: replacing the one or more objects in each frame of the video with a blank; or replacing the one or more objects in each frame of the video with a background color; or replacing the one or more objects in each frame of the video with a background image; 13. The method for processing video according to claim 12.

14. A method for processing video, comprising the steps of: receiving an encoded video and one or more extracted features, wherein one or more objects of the encoded video have been removed and the extracted one or more features are associated with the one or more objects; decoding the encoded video; and generating an image based on the extracted one or more features; and integrating the image and the decoded video based on the extracted one or more features.

13. A receiver system for processing video, comprising:

15. Generating an image based on the extracted one or more features comprises: generating the image by applying the extracted one or more features to the one or more objects.

15. The method for processing video according to claim 14.

Images