Method and display device system for generating extended images from video frames

Abstract

A method and display device system for generating extended images from video frames are disclosed. The display device system includes a processor and a screen. The method is implemented through the display device system and includes: in response to a received video frame, processing the video frame to detect an object therein, and performing a generation algorithm based on the object to generate the extended image; and presenting the video frame and the extended image, wherein the extended image serves as an extension of the video frame. The content displayed on the screen and the extended image together provide a more natural extension to the original video frame, thus offering a wider and more immersive viewing experience to the audience.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to European Patent Application No. EP24307321, filed on 27 December 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to a method and display device system for generating extended images from self-video frames. Background Technology

[0004] In the field of display devices, offset lighting effects (such as backlighting) have become a popular feature, increasing immersion while reducing eye strain. When a display device shows an image (such as from a video or television signal), its lighting module projects a backlight signal onto a surface behind the device (such as a wall). The backlight signal is typically projected outwards from the edge of the screen, and from the viewer's perspective, the projected backlight signal appears to wrap around the screen and seem as an extension of it. Therefore, the backlight signal can enhance video frames to provide a more immersive experience for the viewer. The video frames mentioned below are for illustrative purposes only and are not limiting; frames presented on a display device include any multimedia content (such as video regardless of source), images (regardless of source), or graphics (such as user interfaces).

[0005] Typically, a display device generates a backlight signal based on the video frame currently being displayed on the device. For example, the display device may map each boundary of the video frame being presented to the edge of the screen, detect various features of the boundary (such as color, brightness, texture, or orientation), and generate a backlight signal by mimicking the features detected at the boundary. Therefore, the projected backlight signal can be used to "mimic" a portion of the boundary of a video frame. Summary of the Invention

[0006] Therefore, one object of the present invention is to provide a method for generating augmented images from self-video frames, which aims to provide a better experience for viewers.

[0007] According to an embodiment of the present invention, a method for generating an extended image from a video frame is provided. The method is implemented by a display device system and includes: processing the video frame in response to a received video frame to detect an object therein, and performing a generation algorithm based on the object to generate the extended image; and presenting the video frame and the extended image, wherein the extended image is an extension of the video frame.

[0008] In some embodiments, the display device system includes a screen and a backlight projection unit configured to project light onto the screen, wherein the presented video frame and the extended image include presenting the video frame on the screen and presenting the extended image using the backlight projection unit such that the extended image is presented outside the video frame.

[0009] In some embodiments, the backlight projection unit includes a plurality of backlight projection elements, each backlight projection element projects light outward to a corresponding portion of the screen, generating the expanded image includes generating a plurality of expanded sub-images, the presented video frame and the expanded image include presenting the video frame on the screen and presenting the expanded sub-images respectively using the backlight projection elements, and the expanded image is presented around the video frame.

[0010] In some embodiments, generating the augmented image includes: performing a shape detection operation on the video frame to determine whether the target object is completely contained within the video frame; if the target object is not completely contained within the video frame, generating a supplement in the augmented image, the supplement corresponding to the position of the target object in the video frame; when the supplement is generated, the target object and the supplement together form a composite when the video frame and the augmented image are presented, and the shape of the composite is similar to the target object; and when processing the video frame, it further includes determining that the detected object is the target object and / or background of the video frame, and the generation of the augmented image includes applying the background of the video frame.

[0011] In some embodiments, the method for generating augmented images from video frames further includes: in response to receiving a video comprising a plurality of consecutive video frames, for each consecutive video frame, processing the video frame to detect a target therein, and performing a generation algorithm based on the target to generate the augmented image; detecting a movement pattern of the target based on the video frame; for each consecutive video frame, determining whether the target has moved based on the movement pattern of the target such that at least a portion of the target is not included in the video frame, and if so, generating a supplement in the augmented image corresponding to the position of the target in the video frame; and when the supplement is generated, such that when the video frame and the augmented image are presented, the target and the supplement together form a composite, and the overall shape of the composite is similar to the target.

[0012] In some embodiments, the method for generating augmented images from video frames further includes: for each consecutive video frame, determining whether the target object has completely moved out of the video frame; if so, generating the complement having the shape of the composite in the augmented image.

[0013] In some embodiments, the method for generating augmented images from video frames further includes: determining whether the target object indicates a morphological pattern; and, if the target object indicates a morphological pattern, generating a supplement in the augmented image based on the morphological pattern.

[0014] In some embodiments, the process of generating the augmented image includes generating an appendage that is not presented in the video frame based on the target, and generating the augmented image including the appendage.

[0015] Another object of the present invention is to provide a display device system that can be used to implement the above-described method.

[0016] According to an embodiment of the present invention, a display device system for generating extended images from video frames is provided, the display device system comprising a processor and a screen. In response to a received video frame, the processor processes the video frame to detect an object therein, and performs a generation algorithm based on the object to generate the extended image. The processor controls the screen to display the video frame and the extended image, wherein the extended image is an extension of the video frame.

[0017] In some embodiments, the display device system for generating extended images from video frames further includes a backlight projection unit configured to project light onto the screen, characterized in that the processor further controls the backlight projection unit to present the extended image such that the extended image is presented outside the video frame.

[0018] In some embodiments, the backlight projection unit includes a plurality of backlight projection elements, each backlight projection element projecting light outward onto a corresponding portion of the screen; and the processor generates the extended image by generating a plurality of extended sub-images, and controls the backlight projection unit to present the extended sub-images using the backlight projection elements respectively, such that the extended image is presented around the video frame.

[0019] In some embodiments, the processor generates the augmented image by: performing shape detection operations on the video frame to determine whether the target object is completely contained within the video frame; if the target object is not completely contained within the video frame, generating a supplement in the augmented image, the supplement corresponding to the position of the target object in the video frame; and when the supplement is generated, the target object and the supplement together form a composite when the video frame and the augmented image are presented, and the overall shape of the composite is similar to that of the target object.

[0020] In some embodiments, the processor further performs the following operations: in response to receiving a video comprising a plurality of consecutive video frames, for each consecutive video frame, processing the video frame to detect a target object therein, and performing a generation algorithm based on the target object to generate the augmented image; detecting a movement pattern of the target object based on the video frame; for each consecutive video frame, determining whether the target object has moved based on the movement pattern of the target object such that at least a portion of the target object is not included in the video frame, and if so, generating a supplement in the augmented image corresponding to the position of the target object in the video frame; and when the supplement is generated, such that when the video frame and the augmented image are presented, the target object and the supplement together form a composite, and the overall shape of the composite is similar to the target object.

[0021] In some embodiments, the processor further performs the following operations: for each consecutive video frame, determining whether the target object has completely moved out of the video frame, and if so, generating the complement having the shape of the composite in the augmented image.

[0022] In some embodiments, the processor further performs the following operations: determining whether the target indicates a morphological pattern; and if the target indicates a morphological pattern, generating a complement in the augmented image based on the morphological pattern.

[0023] The beneficial effects of the present invention are as follows: The method and display device system for generating extended images from visual content presented by a display device provide a more natural extension range to the original video frame by the content displayed on the screen and the extended image together, thereby providing a wider and more immersive viewing experience to the audience. Attached Figure Description

[0024] Other features and effects of the present invention will be clearly presented in the embodiments with reference to the accompanying drawings, wherein:

[0025] Figure 1 This is a block diagram illustrating, by way of example, a display device system configured to generate extended images from video frames according to an embodiment of the present invention;

[0026] Figure 2 This describes an exemplary configuration in which there is a display device and four assemblies for projecting light onto the four outer edge backlight projection devices of the display device respectively;

[0027] Figure 3 This is a flowchart illustrating a method for generating augmented images from video frames according to an embodiment of the present invention.

[0028] Figure 4The description includes exemplary video frames displayed on the screen, as well as exemplary augmented images generated therefrom;

[0029] Figure 5 This is a flowchart illustrating some exemplary methods of operation in steps 202 and 204 of this embodiment of the invention;

[0030] Figure 6 This describes an exemplary method for performing shape detection operations and various operations on some consecutive video frames;

[0031] Figure 7 This describes another way of performing shape detection operations and various operations on the video frame. Detailed Implementation

[0032] Before describing the invention in more detail, it should be noted that, where deemed appropriate, reference numerals are repeatedly used in the drawings to indicate corresponding or similar components, which may have similar characteristics.

[0033] Throughout this specification, the terms “coupled to” or “connected to” can refer to a direct connection between multiple electrical devices / equipment / instruments via a conductive material (e.g., wires), an indirect connection between two electrical devices / equipment / instruments via one or more other devices / equipment / instruments, or wireless communication.

[0034] Figure 1 This is a block diagram illustrating, by way of example, a display device system 100 configured to generate extended images from video frames according to an embodiment of the present invention.

[0035] The display device system 100 includes a display device 110 and a backlight projection unit 120.

[0036] The display device 110 includes a processor 112, a storage module 114, a communication unit 116, and a screen 118.

[0037] The processor 112 may include, but is not limited to, a single-core processor, a multi-core processor, a dual-core mobile processor, a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or a radio frequency integrated circuit (RFIC), etc.

[0038] The storage module 114 can be implemented using, for example, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), firmware, and / or flash memory. The storage module 114 stores a software application program containing a number of instructions, which, when executed by the processor 112, cause the processor 112 to perform the operations described below.

[0039] The communication unit 116 may include one or more of the following: a radio frequency integrated circuit (RFIC), a short-range wireless communication module supporting short-range wireless communication networks using wireless technologies such as Bluetooth and / or Wi-Fi, and a mobile communication module supporting telecommunications technologies such as Long Term Evolution (LTE), 3G, 4G, or 5G. The display device 110 is configured to use the communication unit 116 to communicate with the backlight projection unit 120, a cloud server 130 (in some embodiments), or a network (e.g., the Internet).

[0040] The processor 112 is connected to the storage module 114, the communication unit 116, and the screen 118, and the screen 118 can be implemented using a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or other suitable display devices. In use, the screen 118 can receive an input signal carrying visual information. The input signal can be in various forms, such as a television signal, video or images from an external source (e.g., via a High Definition Multimedia Interface (HDMI) or Universal Serial Bus (USB) port), or video or images from a network received via the communication unit 116 (e.g., online video platforms like Youku). The screen 118 can be controlled by the processor 112 to display the input signal or the video. Notably, the input signal or the video can be in the form of multiple consecutive video frames. The term "video frame" hereafter refers to a frame containing any multimedia content displayed on the screen 118, such as video (regardless of source), images (regardless of source), or graphical representations (e.g., a user interface). In use, the screen 118 is typically positioned in front of a wall.

[0041] The backlight projection unit 120 includes one or more light sources 122. Figure 1In this embodiment, four light sources 122 (labeled 122A, 122B, 122C, and 122D, respectively) are presented. The light sources 122 are connected to the processor 112 via the communication unit 116 and are configured to project light outwards (e.g., onto a wall behind the screen 118). Each light source 122 can be implemented using a commercially available projector or other non-projection light source capable of projecting a specific light pattern onto a wall. Notably, in some embodiments, the backlight projection unit 120 can be integrated into the back of the screen 118.

[0042] For example, such as Figure 2 As shown, the screen 118 has four edges (labeled 118A, 118B, 118C, and 118D respectively), and the four light sources 122 can be combined to project light onto predetermined portions of the wall near the edges. It is worth noting that, in Figure 2 The configuration shown is only an exemplary configuration. In other embodiments, different numbers of light sources 122 can be used to project light in different directions (e.g., different areas on the wall).

[0043] The cloud server 130 is connected to the display device 110 and can provide software applications for the display device 110 to use in this invention. It is worth noting that in some embodiments, the display device 110 can perform the operations described below without connecting to the cloud server 130, and may connect to the cloud server to update the software applications. Alternatively, the display device 110 can be implemented without connecting to the cloud server 130, and the software applications can be obtained through other means.

[0044] In use, when the display device 110 is enabled, the processor 112 receives the television signal or the video and controls the screen 118 to display the television signal or the video in a continuous image format. When the screen 118 displays the television signal or the video, the user can operate an interface (e.g., via a remote control, not shown) to activate a backlight projection function. Alternatively, the backlight projection function can be preset to activate when the display device 110 is turned on.

[0045] In response to the activation of the backlight projection function, the processor 112 may execute a software application stored in the storage module 114 to perform some operations to generate a backlight projection signal, which enables the backlight projection unit 120 to project an extended image (e.g. on a wall) to provide a backlight effect.

[0046] Figure 3This is an embodiment of the method 200 for generating augmented images from video frames according to the present invention. In this embodiment, the method 200 is achieved by... Figure 1 The display device system 100 shown is implemented in this way.

[0047] In step 202, in response to the received signal to enable the backlight projection function, the processor 112 processes the video frame from the television signal or the video to detect at least one target therefrom.

[0048] Since the backlight projection function is preset to start when the display device 110 is turned on, the method will start as soon as possible when the display device 110 is turned on.

[0049] Figure 4 The description shows an exemplary video frame displayed on screen 118. In this embodiment, the video frame includes water, mountains, multiple trees, multiple clouds, and sky. The reflection of the mountains can also be seen on the water. During the processing of the video frame, the processor 112 can detect whether objects in the video frame are used as targets and / or background. For example, in Figure 4 In the video frame, the sky can serve as the background, and the mountains can simultaneously serve as both the subject and the background. In this embodiment, the processor 112 also determines multiple features of the background of the video frame. In this embodiment, the features may include color, brightness, texture, or direction, etc.

[0050] Next, in step 204, the processor 112 executes a generation algorithm based on at least one of the targets to generate an augmented image.

[0051] In different embodiments, the expanded image can be projected onto different portions of the wall outside the screen 118 to achieve the backlighting effect. Figure 4 In some embodiments, the augmented image can be composed of multiple augmented sub-images corresponding to the four edges of the screen 118. In some embodiments, the augmented image may correspond to only one edge of the screen 118.

[0052] In some embodiments, the television signal or the video frame of the video may have a different aspect ratio than the screen 118 (e.g., when a movie shot in a 16:9 aspect ratio is played on the screen 118 with a 4:3 aspect ratio). When the video frame is played on the screen 118, one or two blank spaces (e.g., blank spaces at the top and bottom) may be inserted around the perimeter of the video frame. In this case, the extended image may be generated corresponding to one or two blank spaces to provide the backlight effect to "extend" the screen 118.

[0053] In different implementations, step 202 may include some different methods. Figure 5 This is a flowchart illustrating some exemplary methods of steps 202 and 204 in an embodiment of the present invention. It is worth noting that in other embodiments, the operation in step 202 may only include the steps shown below. Figure 5 One or more of the methods.

[0054] In the "shape detection" mode, the processor 112 performs a shape detection operation on a video frame to attempt to determine the shape of an object in the video frame. In some embodiments, the shape detection operation can be performed by extracting metadata of the video frame and executing an edge detection algorithm to determine the outline of each object in the video frame.

[0055] exist Figure 6 In one example shown, by performing the shape detection operation in one of the video frames (e.g., frame N-5), the processor 112 may detect a partially circular object (e.g., the sun).

[0056] In the "motion detection" mode, the processor 112 performs the aforementioned shape detection operations on a series of consecutive video frames to determine whether an object is "moving." That is, for the same object in each of the video frames, the processor 112 determines the object's movement pattern based on the video frames. Figure 6 In the example, six consecutive video frames are presented (sequentially labeled frames N-5 to N), and the processor 112 performs the shape detection operation described above on each of the consecutive video frames. The processor 112 can further perform a dynamic detection algorithm on the consecutive video frames to determine whether the object is moving based on the different locations of the same object in the consecutive video frames. Figure 6 In the example, it can be determined that the circular object is rising and is about to move out of the video frame.

[0057] In the "shape detection" method, after detecting an object, the processor 112 determines whether the object has a shape pattern. Specifically, the term "shape pattern" refers to an object that is a continuous object extending beyond the video frame and having a specific repeating shape. For example, the object could be a wooden fence comprising multiple vertical planks spaced apart from each other, with multiple horizontal planks intersecting the vertical planks. In this case, the object can be said to have a shape pattern.

[0058] In the "additional object determination" method, after detecting an object, the processor 112 executes a machine learning algorithm to determine whether there are any additional objects that should appear outside the boundaries of the video frame. For example, if the video frame includes a portion of a baseball field with first, second, and third bases, the processor 112 might determine that there is a home plate that should be located outside the boundaries of the video frame. In other examples, when the video frame includes the sky, the processor 112 might determine whether there might be additional clouds, birds, or other objects that could appear outside the boundaries of the video frame.

[0059] After performing one or more steps 202 in these manner, the processor 112 determines which detected objects should be used as objects for generating the augmented image to be projected. In some embodiments, typically at least one target object is present for each video frame.

[0060] Figure 6 An exemplary manner in which step 202 is performed on the video frame is described. Figure 7 This describes another way to perform step 202. Specifically, in Figure 7 In this example, the processor 112 executes one or more pre-trained neural networks to perform the operation of step 202. The training can be accomplished by preparing some reference video frames and some cropped video frames that are cut from the reference video frames. The neural network then attempts to generate an augmented image using the cropped video frames, making a cropped video frame and the augmented image cooperatively resemble the corresponding reference video frame. That is, when attempting to generate an augmented image from its own cropped video frames, the corresponding reference video frame can be used as a reference.

[0061] Generally speaking, Figure 6 Examples can be used when video frames have relatively low resolution, while Figure 7 Examples can be used when video frames have relatively high resolution, but actual implementations are not limited to this.

[0062] Next, in step 204, the processor 112 executes a generation algorithm based on at least one of the targets to generate an augmented image.

[0063] exist Figure 5 In the example, step 204 also includes several methods, corresponding to those in step 202. Specifically, for an object in a video frame, after completing the shape detection of the object in step 202, in step 204, the processor 112 determines whether the object is completely contained within the video frame. Figure 6In the example, the circular object is not fully contained in the video frame labeled N-1. In this case, the processor 112 generates a supplementary object corresponding to the complete object and located in a portion of the augmented image at the position corresponding to the object within the video frame. Generally, the shape of the supplementary object will differ from the shape of the object.

[0064] In some embodiments, the supplement is generated such that when the video frame and the augmented image are presented by the display device 110, the target and the supplement together form a composite with a shape similar to the complete target. Figure 6 In the example, the extended image only covers the edge 118A of the screen 118. For the video frame marked as frame N-1, the supplement is to complete the circular object that is moving outward from the edge of the video frame, so as to form a complete circular object with the target object. Figure 4 In the example, for the (incomplete) mountain range in the video frame, the complement is a portion of the mountain range that can be generated to extend the mountain range to the left side of the video frame.

[0065] After determining that the target is moving through the "motion detection" method in step 202, the processor 112 determines whether the target is moving based on the movement pattern of the target so that part of the target is no longer included in the video frame being processed.

[0066] In determining the subject matter (e.g.) Figure 6 The processor 112 generates a supplementary object corresponding to the target object and located in part of the augmented image corresponding to the position of the target object in the video frame, where only a portion of the circular object is presented in the video frame and is about to move out of the video frame.

[0067] For example, in Figure 6 In the example, the target object (the circular object) is only partially presented in the video frames marked from frame N-5 to frame N-1, and is moving out from the top of the video frame. For each video frame, the processor 112 generates a different complement to (at least partially) complement the circular object and coordinate with the movement of the circular object so that the circular object is moving out of the video frame from the screen 118, that is, moving to the augmented image and further away.

[0068] Additionally, when it is determined that the target object has completely moved out of the video frame, the processor 112 generates a supplement in the augmented image, the supplement having the shape of the composite or part of the object that last "left" the video frame.

[0069] For example, in Figure 6 In the example, in the video frame labeled as frame N, the circular object no longer exists (i.e., the circular object has completely moved out of the video frame). In this case, the processor 112 generates a complete circular object (in the augmented image) to be presented above the video frame. In subsequent video frames, the processor 112 can generate the complete circular object at different locations to present the circular object as continuously moving in the motion mode.

[0070] If, in step 202, the target object indicates a morphological pattern, the processor 112 generates the supplementary element in the augmented image based on the morphological pattern. In this example, when a wooden fence is detected, the processor 112 can generate a supplementary element extending into the augmented image in the form of another portion of the wooden fence.

[0071] After determining in step 202 that the object should be located outside the boundary of the video frame, the processor 112 generates an object not shown in the video frame based on the target object, and generates an augmented image containing the object. For example, if the video frame includes a portion of a baseball field with first, second, and third bases, the processor 112 can generate home base, and the position of home base is generated based on the positions of first, second, and third bases. In other examples, if the video frame includes the sky, the processor 112 can generate additional clouds, birds, or other objects included in the augmented image.

[0072] It is worth noting that in some embodiments, the background of the video frame can also be applied during the generation of the image. Figure 4 For example, the extended image 119 comprises four sub-images (labeled 119A, 119B, 119C, and 119D, respectively), and the four sub-images surround the video frame. Since each of the sub-images is a portion of the background (e.g., sky, water, or mountains) extending from the video frame, the background should also be included in the sub-image when it is generated, in addition to the objects (if any). For example, sub-image 119A should include a background with the sky, sub-images 119B and 119D should include a background with the sky, mountains, and water, and sub-image 119C should include a background with the water.

[0073] It is worth noting that in some embodiments, other data related to the video frame can also be applied during the generation of the augmented image. For example, when the video frame is extracted from a video, metadata and / or audio associated with the video frame may also be utilized. By using the metadata, the processor 112 can determine information about the video containing the video frame and determine whether to adjust the features of the augmented image. For example, if there is a musical beat in the video at the time the video frame is located, the augmented image can be generated with higher brightness.

[0074] Next, in step 206, the processor 112 presents both the video frame and the extended image, wherein the extended image serves as an extension of the video frame. Specifically, in some embodiments, the processor 112 may control the screen 118 to present the video frame and control the backlight projection unit 120 to present the extended image. In some other embodiments, when the screen 118 is not fully used to display the video frame and the screen 118 has at least one blank area, the processor 112 may generate the extended image to fill the at least one blank area and control the screen 118 to also display the extended image.

[0075] The augmented image is generated using the method described above, and when presented to the user along with the video frame, it provides a more satisfactory viewing experience.

[0076] It is worth noting that in the above operations, the algorithm or neural network can be obtained using commercially available products.

[0077] It is worth noting that, in some embodiments, the above-described operating method can be run by a post-processor separate from the processor 112.

[0078] In short, embodiments of the present invention provide a method for generating extended images from video frames and a display device system 100. The content displayed on the screen 118 and the extended images together provide a more natural extension of the original video frame, thereby providing a wider and more immersive viewing experience to the audience.

[0079] In the foregoing description, numerous specific details have been set forth for purposes of explanation in order to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that one or more other embodiments may be practiced without some of these specific details. It should also be understood that references throughout the specification to “an embodiment,” “embodiment,” or ordinal numbers, etc., imply that a particular feature, structure, or characteristic may be included in an implementation of this disclosure. It should be further understood that, in the specification, various features are sometimes combined in a single embodiment, drawing, or description thereof to simplify the disclosure and aid in understanding various inventive forms, and one or more features, or, where appropriate, details from one embodiment, may be implemented together with one or more features or details from another embodiment in an implementation of this disclosure.

[0080] However, the above description is merely an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the claims and description of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for generating augmented images from self-video frames, implemented via a display device system, characterized in that, The method for generating augmented images from video frames includes: In response to a received video frame, the video frame is processed to detect a target therein, and a generation algorithm is executed based on the target to generate the augmented image; and The video frame and the extended image are presented, wherein the extended image is an extension of the video frame.

2. The method for generating augmented images from self-video frames according to claim 1, characterized in that, The display device system includes a screen and a backlight projection unit configured to project light onto the screen. The presented video frame and the extended image include presenting the video frame on the screen and presenting the extended image using the backlight projection unit, such that the extended image is presented outside the video frame.

3. The method for generating augmented images from self-video frames according to claim 2, characterized in that, The backlight projection unit includes multiple backlight projection elements, each of which projects light outward to a corresponding portion of the screen. Generating the expanded image includes generating multiple expanded sub-images. The presented video frame and the expanded image include presenting the video frame on the screen and using the backlight projection elements to present the expanded sub-images respectively, and the expanded image is presented around the video frame.

4. The method for generating augmented images from self-video frames according to claim 1, characterized in that, Generating the augmented image includes: Perform shape detection operations on the video frame to determine whether the target object is completely contained within the video frame; If the target object is not fully contained in the video frame, a supplement is generated in the augmented image, and the supplement corresponds to the position of the target object in the video frame; When the supplementary element is generated, when the video frame and the augmented image are presented, the target object and the supplementary element together form a composite, and the shape of the composite is similar to that of the target object; and The processing of the video frame further includes determining that the detected object is the target object and / or background of the video frame, and the generation of the augmented image includes applying the background of the video frame.

5. The method for generating augmented images from self-video frames according to claim 1, characterized in that, The method for generating augmented images from video frames further includes: In response to receiving a video containing multiple consecutive video frames, for each consecutive video frame, the video frame is processed to detect a target therein, and a generation algorithm is performed based on the target to generate the augmented image; The movement pattern of the target is detected based on the video frames; For each consecutive video frame, it is determined whether the target object has moved based on the movement pattern of the target object, such that at least a portion of the target object is not included in the video frame. If so, a supplement is generated in the portion of the augmented image corresponding to the position of the target object in the video frame. and When the complement is generated, when the video frame and the augmented image are presented, the target object and the complement together form a composite, and the overall shape of the composite is similar to that of the target object.

6. The method for generating augmented images from self-video frames according to claim 5, characterized in that, The method for generating augmented images from video frames further includes: For each consecutive video frame, determine whether the target object has completely moved out of the video frame; if so, generate the complement having the shape of the composite in the augmented image.

7. The method for generating augmented images from self-video frames according to claim 1, characterized in that, The method for generating augmented images from video frames further includes: Determine whether the object indicates a morphological pattern; and When the target object indicates a morphological pattern, a complement is generated in the augmented image based on the morphological pattern.

8. The method for generating augmented images from self-video frames according to claim 1, characterized in that, The process of generating the augmented image includes generating an addendum that is not presented in the video frame based on the target object, and generating the augmented image including the addendum.

9. A display device system for generating extended images from video frames, the display device system comprising a processor and a screen, characterized in that: In response to a received video frame, the processor processes the video frame to detect a target therein, and performs a generation algorithm based on the target to generate the augmented image; and The processor controls the screen to display the video frame and the extended image, wherein the extended image is an extension of the video frame.

10. The display device system for generating extended images from self-video frames according to claim 9, characterized in that, It also includes a backlight projection unit assembled to project light onto the screen, and the processor further controls the backlight projection unit to present the extended image, such that the extended image is presented outside the video frame.

11. The display device system for generating extended images from self-video frames according to claim 10, characterized in that: The backlight projection unit includes multiple backlight projection elements, each of which projects light outward onto a corresponding portion of the screen; and The processor generates the extended image by generating multiple extended sub-images, and controls the backlight projection unit to present the extended sub-images using the backlight projection elements respectively, so that the extended image is presented around the video frame.

12. The display device system for generating extended images from self-video frames according to claim 9, characterized in that, The processor generates the augmented image through the following operations: Perform shape detection operations on the video frame to determine whether the target object is completely contained within the video frame; If the target object is not fully contained in the video frame, a supplement is generated in the augmented image, and the supplement corresponds to the position of the target object in the video frame; When the complement is generated, when the video frame and the augmented image are presented, the target object and the complement together form a composite, and the overall shape of the composite is similar to that of the target object.

13. The display device system for generating extended images from self-video frames according to claim 9, characterized in that, The processor also performs the following operations: In response to receiving a video containing multiple consecutive video frames, for each consecutive video frame, the video frame is processed to detect a target therein, and a generation algorithm is performed based on the target to generate the augmented image; The movement pattern of the target is detected based on the video frames; For each consecutive video frame, it is determined whether the target object has moved based on the movement pattern of the target object, such that at least a portion of the target object is not included in the video frame. If so, a supplement is generated in the portion of the augmented image corresponding to the position of the target object in the video frame. and When the complement is generated, when the video frame and the augmented image are presented, the target object and the complement together form a composite, and the overall shape of the composite is similar to that of the target object.

14. The display device system for generating extended images from self-video frames according to claim 13, characterized in that, The processor also performs the following operations: For each consecutive video frame, determine whether the target object has completely moved out of the video frame; if so, generate the complement having the shape of the composite in the augmented image.

15. The display device system for generating extended images from self-video frames according to claim 9, characterized in that, The processor also performs the following operations: Determine whether the object indicates a morphological pattern; and When the target object indicates a morphological pattern, a complement is generated in the augmented image based on the morphological pattern.

Images