Image expansion method and device

Abstract

The invention relates to an image expansion method and device, and relates to the technical field of image processing, and the method comprises the steps: obtaining an initial image to be subjected to image expansion and image parameters of the initial image; obtaining screen parameters of the target display equipment; determining an expansion area corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge proportion; and performing image expansion on the initial image based on the expansion region to generate an expanded image of the initial image. According to the invention, image expansion is carried out without losing image details, the image quality is improved, and the expansion area is determined according to the screen parameters of the target display device, so that the image can adapt to different devices, the problem that a black filling area appears when the image is displayed on different devices is avoided, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image expansion method and device. Background Art

[0002] In modern society, images have become one of the main ways for people to obtain information and entertainment. There are many devices for viewing images, including TVs, mobile phones, tablets and other devices. Different device screen ratios will cause the size and position of images displayed on different devices to be inconsistent. For example, when viewing a picture on TV, a black filling area may appear, but not when viewing it on a mobile phone or tablet. This is because the ratio of TV screens is usually 16:9 or 4:3, while the ratios of mobile phone and tablet screens are more diverse, which may be 18:9, 21:9, 1080p, 2K, 4K, etc. Therefore, if a picture is enlarged directly to watch on TV, a black filling area will appear, affecting the display effect of the image. Secondly, if the image is enlarged and displayed, the image content will be lost, affecting the user experience. Summary of the invention

[0003] The present disclosure provides an image expansion method to at least solve the problem of black filling areas appearing when an image is displayed on different devices. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a method for expanding an image is provided, comprising: obtaining an initial image to be expanded and image parameters of the initial image; obtaining screen parameters of a target display device; determining an expansion area corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge ratio; and expanding the initial image based on the expansion area to generate an extended image of the initial image.

[0005] In some embodiments, an initial image is expanded based on an extended area to generate an extended image of the initial image, including: acquiring image features of the initial image, and generating extended text of the initial image in combination with the extended area; inputting the extended text and the initial image into an image extension model to acquire an extended image output by the image extension model.

[0006] In some embodiments, after the initial image is expanded based on the extended area to generate an extended image of the initial image, the method further includes: determining whether the extended image requires super-resolution processing based on image parameters and screen parameters; in response to determining that the extended image requires super-resolution processing, performing super-resolution processing on the extended image to obtain a target image after super-resolution processing; in response to determining that the extended image does not require super-resolution processing, using the extended image as the target image; and displaying the target image on a target display device.

[0007] In some embodiments, obtaining an initial image to be expanded includes: obtaining an initial video stream to be expanded; extracting key frames from the initial video stream and using the extracted key frame images as the initial image; and, after expanding the initial image based on the extended area to generate an extended image of the initial image, further including: sorting the extended images according to the order of the initial images in the initial video stream, and obtaining a sequence of extended images generated after the sorting; processing the extended image sequence based on interpolation technology and video motion compensation technology to generate a compensated image corresponding to each extended image; and combining the extended image sequence with the compensated image to obtain a target video stream generated after the combination.

[0008] In some embodiments, obtaining an initial image to be subjected to image expansion includes: obtaining an initial video stream to be subjected to video expansion; obtaining all video frame images contained in the initial video stream, and taking each video frame image as the initial image; and, after performing image expansion on the initial image based on the extended area to generate an extended image of the initial image, further including: generating a target video stream based on the extended image in the order of the initial image in the initial video stream.

[0009] In some embodiments, the image expansion method further includes: displaying the target images contained in the target video stream on the target display device in sequence.

[0010] According to a second aspect of an embodiment of the present disclosure, there is provided an image expansion device, comprising: a first acquisition module, used to acquire an initial image to be expanded and image parameters of the initial image; a second acquisition module, used to acquire screen parameters of a target display device; an area determination module, used to determine an expansion area corresponding to the initial image based on image parameters, screen parameters and a preset image edge ratio; and an image expansion module, used to expand the initial image based on the expansion area to generate an extended image of the initial image.

[0011] In some embodiments, the image extension module is further used to: obtain image features of the initial image, and generate extended text of the initial image in combination with the extended area; input the extended text and the initial image into the image extension model to obtain the extended image output by the image extension model.

[0012] In some embodiments, the image expansion device also includes: a judgment module, which is used to judge whether the extended image requires super-resolution processing based on image parameters and screen parameters; a first processing module, which is used to perform super-resolution processing on the extended image in response to judging that the extended image requires super-resolution processing to obtain a target image after super-resolution processing; a second processing module, which is used to use the extended image as the target image in response to judging that the extended image does not require super-resolution processing; and a first display module, which is used to display the target image on a target display device.

[0013] In some embodiments, the first acquisition module is further used to: acquire an initial video stream to be extended; perform key frame extraction on the initial video stream, and use the extracted key frame images as initial images; and the image extension module is further used to: sort the extended images according to the order of the initial images in the initial video stream, and acquire an extended image sequence generated after sorting; process the extended image sequence based on interpolation technology and video motion compensation technology, and generate a compensated image corresponding to each extended image; combine the extended image sequence with the compensated image, and acquire the target video stream generated after the combination.

[0014] In some embodiments, the first acquisition module is further used to: acquire an initial video stream to be subjected to video expansion; acquire all video frame images contained in the initial video stream, and use each video frame image as an initial image; and the image expansion module is further used to: generate a target video stream based on the extended image in the order of the initial images in the initial video stream.

[0015] In some embodiments, the image expansion apparatus further includes: a second display module, configured to sequentially display the target images contained in the target video stream on a target display device.

[0016] According to the third aspect of an embodiment of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the image expansion method as described in the embodiment of the first aspect of the present application.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to implement the image expansion method as described in the embodiment of the first aspect of the present application.

[0018] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is proposed, including a computer program, which, when executed by a processor, implements the image expansion method as described in the embodiment of the first aspect of the present application.

[0019] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: the present application performs image expansion without losing image details, improves image quality, determines the expansion area according to the screen parameters of the target display device, enables the image to adapt to different devices, avoids the problem of black fill areas when the image is displayed on different devices, and improves the user experience.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0022] Figure 1 It is a schematic diagram showing an exemplary implementation of an image expansion method according to an exemplary embodiment.

[0023] FIG. 2( a ) is a schematic diagram showing the display of an initial image A on a mobile phone as shown in the present application.

[0024] FIG2( b ) is a schematic diagram showing the display of an expanded image of the initial image A shown in the present application on a mobile phone.

[0025] FIG3( a ) is a schematic diagram showing the display of an initial image B on a mobile phone as shown in the present application.

[0026] FIG3( b ) is a schematic diagram showing the display of an expanded image of the initial image B shown in the present application on a mobile phone.

[0027] Figure 4 It is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application.

[0028] FIG5( a ) is a schematic diagram of a process of generating and displaying an extended image of an initial image C shown in the present application.

[0029] FIG5( b ) is a schematic diagram showing an extended image of an initial image C shown in the present application being displayed on devices of different sizes.

[0030] Figure 6 It is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application.

[0031] FIG. 7( a ) is a schematic diagram of a video shown in the present application in which a black area appears on the screen due to size mismatch during video playback.

[0032] FIG. 7( b ) is a schematic diagram of an extended image generated when a key frame image is extended according to the present application.

[0033] Figure 8 It is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application.

[0034] Fig. 9 It is a schematic diagram of an image expansion device shown in the present application.

[0035] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0036] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0038] Figure 1 is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application, such as Figure 1 As shown, the image expansion method includes the following steps:

[0039] S101, obtaining an initial image to be expanded and image parameters of the initial image.

[0040] An image to be expanded is determined as an initial image. Image parameters of the initial image are obtained, wherein the image parameters at least include an image size and an image resolution of the initial image.

[0041] S102: Acquire screen parameters of a target display device.

[0042] A target display device for displaying the extended image after the initial image is extended is determined, and screen parameters of the target display device are acquired, wherein the screen parameters at least include a screen size and a screen resolution of the target display device.

[0043] Exemplarily, the target display device refers to a device used to display an extended image after the initial image is expanded, including but not limited to laptop computers, mobile phones, tablets, televisions, wearable devices, game consoles and other devices.

[0044] S103, determining an extended area corresponding to the initial image based on image parameters, screen parameters and a preset image edge ratio.

[0045] Figure 2(a) is a schematic diagram of the display of an initial image A on a mobile phone shown in the present application. As shown in Figure 2(a), when the initial image A is displayed on the mobile phone screen, since the image size does not match the mobile phone screen size, black areas appear on the left and right sides of the mobile phone screen when displaying the initial image A. In order to avoid this situation, the present application determines the extended area corresponding to the initial image based on the image parameters, screen parameters and the preset image edge ratio. Taking Figure 2(a) as an example, it can be regarded as that the black areas on the left and right sides of the initial image A in Figure 2(a) are used as the extended areas corresponding to the initial image A.

[0046] S104: Expand the initial image based on the expansion area to generate an expanded image of the initial image.

[0047] After determining the extended area corresponding to the initial image, it is necessary to fill the extended area with appropriate content according to the content of the initial image to complete the entire image. Optionally, the extended area of ​​the initial image can be filled based on the Stable Diffusion model to generate an extended image of the initial image.

[0048] Figure 2(b) is a schematic diagram of the display of the extended image of the initial image A shown in the present application on the mobile phone. As shown in Figure 2(b), the extended image generated after the black areas on the left and right sides of the initial image A are expanded and filled can fill the entire mobile phone screen, and the content filled in the extended area is based on the characteristics of the initial image A itself, making the extended image of the initial image A fuller and more harmonious.

[0049] To further facilitate understanding, Figure 3(a) is a schematic diagram of the display of an initial image B on a mobile phone shown in the present application. As shown in Figure 3(a), the top of the initial image B is displayed as a black area on the mobile phone screen, and the black area is used as an extended area to expand the initial image to generate an extended image of the initial image B. Figure 3(b) is a schematic diagram of the display of the extended image of the initial image B shown in the present application on a mobile phone. As shown in Figure 3(b), the generated extended image of the initial image B can fill the entire mobile phone screen, and the content filled in the extended area is filled based on the characteristics of the initial image B itself, so that the extended image of the initial image B is fuller and more harmonious.

[0050] The embodiment of the present application proposes a method for expanding an image, which includes obtaining an initial image to be expanded and image parameters of the initial image; obtaining screen parameters of a target display device; determining an expansion area corresponding to the initial image based on the image parameters, screen parameters, and a preset image edge ratio; and expanding the initial image based on the expansion area to generate an expanded image of the initial image. The present application expands the image without losing image details, improves image quality, determines the expansion area according to the screen parameters of the target display device, enables the image to adapt to different devices, avoids the problem of black filling areas when the image is displayed on different devices, and improves the user experience.

[0051] Figure 4 is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application, such as Figure 4 As shown, the image expansion method includes the following steps:

[0052] S401, obtaining an initial image to be expanded and image parameters of the initial image.

[0053] S402: Acquire screen parameters of the target display device.

[0054] S403, determining an extended area corresponding to the initial image based on image parameters, screen parameters and a preset image edge ratio.

[0055] Regarding the specific implementation of steps S401 to S403, reference may be made to the specific introduction of the relevant parts in the above embodiments, which will not be elaborated here.

[0056] S404, acquiring image features of the initial image, and combining them with the extended area to generate extended text of the initial image.

[0057] Computer vision technology, such as convolutional neural network (CNN) or deep learning model, can be used to extract features of the initial image, obtain image features of the initial image, and generate extended text of the initial image in combination with the extended area. The extended text is used to describe the expansion principle when expanding the extended area of ​​the initial image.

[0058] Exemplarily, as shown in FIG3(a), the top of the initial image B is displayed as a black area on the mobile phone screen, and adjacent to the black area is the blue sky and white clouds displayed on the image. The black area is used as an extended area to expand the initial image, and the corresponding extended text is "further fill the blue sky and white clouds above the initial image B."

[0059] S405: Input the extended text and the initial image into an image extension model, and obtain an extended image output by the image extension model.

[0060] Optionally, the image extension model can use a Stable Diffusion model to understand the initial image content through Stable Diffusion and adaptively extend the initial image. Stable Diffusion mainly consists of a text encoder, a UNet+ scheduler, and an automatic encoder-decoder.

[0061] The expanded text and the initial image are input into the text encoder. The text encoder is a Transformer language model. As a language understanding component, the Transformer language model can accept text prompt words and generate text vectors (token embeddings).

[0062] UNet+Scheduler: Processes information step by step in the information (latent) space. The input is the noise consisting of token embeddings and an initialized multidimensional number, and the output is an array of processed information.

[0063] The autoencoder decoder uses the processed information array to draw the final image. The input is the processed information array (dimension: (4, 64, 64)) and the output is the generated image (dimension: (3, 512, 512), i.e. (red / green / blue; width, height)). The image information creator runs multiple steps to generate the image information. The image information creator operates entirely in the image information space, which is faster than the previous diffusion model that operates in the pixel space. The image information creator processes the information step by step so that the image decoder can then produce a high-quality image.

[0064] Exemplarily, the extended text corresponding to FIG3(a) is “further fill the blue sky and white clouds above the initial image B”. After image expansion, the extended image of the initial image B shown in FIG3(b) is obtained.

[0065] S406: Determine whether the extended image needs super-resolution processing based on the image parameters and the screen parameters.

[0066] The image resolution of the initial image is compared with the screen resolution of the target display device to determine whether the extended image requires super-resolution processing.

[0067] S407: In response to determining that the extended image needs to be subjected to super-resolution processing, super-resolution processing is performed on the extended image to obtain a target image obtained after the super-resolution processing.

[0068] If it is determined that the extended image needs super-resolution processing, for example, the resolution of the original image is low and the extended image needs to be displayed on a high-resolution screen, super-resolution processing is performed on the extended image to obtain a target image after super-resolution processing.

[0069] S408: In response to determining that the extended image does not need to be subjected to super-resolution processing, the extended image is used as a target image.

[0070] S409, displaying the target image on the target display device.

[0071] Figure 5(a) is a schematic diagram of the generation and display process of an extended image of an initial image C shown in the present application. As shown in Figure 5(a), if the initial image C is intended to be displayed on a display screen, but the initial image C is different in size from the display screen, then based on the initial image C, the extended image of the initial image C is generated, and then the extended image of the initial image C can be appropriately displayed on the display screen.

[0072] For example, if the initial image C is to be set as the wallpaper on various mobile devices, in order to avoid black edges or inappropriate sizes when the wallpaper is displayed on different devices, an extended image of the initial image C can be obtained to crop the extended image of the initial image C to different sizes to adapt to various electronic devices. FIG5(b) is a schematic diagram of the extended image of the initial image C shown in the present application being displayed on devices of different sizes. As shown in FIG5(b), based on the extended image of the initial image C, it can be properly displayed on different devices.

[0073] By acquiring the initial image to be expanded and the image parameters of the initial image, the embodiment of the present application can ensure that the original image has been fully processed and analyzed before the image expansion is performed; by acquiring the screen parameters of the target display device, it can ensure that the expanded image can be correctly displayed on the target device and will not be distorted or blurred due to issues such as resolution or size; by determining the expansion area corresponding to the initial image based on the image parameters, screen parameters and a preset image edge ratio, it can ensure that the expanded image has a similar visual effect to the original image while fully utilizing the display space of the target device; by acquiring the image features of the initial image and combining the expansion area to generate the expanded text of the initial image, more detailed information can be provided for subsequent image expansion. information and description; by inputting the extended text and the initial image into the image extension model and obtaining the extended image output by the image extension model, the quality and accuracy of the extended image can be further improved; by judging whether the extended image needs super-resolution processing based on the image parameters and the screen parameters, unnecessary calculation and processing can be avoided, thereby improving the efficiency and performance of the system; if it is judged that the extended image needs super-resolution processing, the extended image can be super-resolution processed to obtain the target image obtained after the super-resolution processing, thereby further improving the quality and accuracy of the extended image; if it is judged that the extended image does not need super-resolution processing, the extended image can be used as the target image and directly displayed on the target display device, thereby further improving the efficiency and performance of the system.

[0074] Figure 6 is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application, which is applicable to the scenario of performing image expansion on the entire video stream, such as Figure 6 As shown, the image expansion method includes the following steps:

[0075] S601: Acquire an initial video stream to be extended.

[0076] In addition to directly expanding the image, it is common that the video size often does not match the screen size of the playback device. FIG7(a) is a schematic diagram of a video playback in this application, in which a black area appears on the screen due to size mismatch, so the video stream also needs to be expanded. In this application, the video stream to be expanded is used as the initial video stream.

[0077] S602: extract key frames from the initial video stream, and use the extracted key frame images as initial images.

[0078] Since the initial video stream is composed of a large number of image frames, and the image expansion calculation for all image frames is too large, in this application, key frame extraction is performed on the initial video stream, and the extracted key frame image is used as the initial image. Common key frame extraction methods include motion vector-based detection methods, optical flow-based detection methods, etc.

[0079] S603: Acquire an initial image to be expanded and image parameters of the initial image.

[0080] S604: Acquire screen parameters of the target display device.

[0081] S605: Determine an extended area corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge ratio.

[0082] S606: Expand the initial image based on the expansion area to generate an expanded image of the initial image.

[0083] Regarding the specific implementation of steps S603 to S606, reference may be made to the specific introduction of the relevant parts in the above embodiments, which will not be elaborated here.

[0084] Figure 7(b) is a schematic diagram of an extended image generated when a key frame image is expanded as shown in the present application. As shown in Figure 7(b), after the key frame image corresponding to Figure 7(a) is expanded, there is no longer a black area on the screen, and the extended image fills the entire display screen.

[0085] S607, sorting the extended images according to the order of the initial images in the initial video stream, and obtaining a sequence of extended images generated after sorting.

[0086] According to the frame number, timestamp and other data of each initial image, the order of each initial image in the initial video stream is determined, the extended images are sorted according to the order of the initial images in the initial video stream, and the extended image sequence generated after sorting is obtained.

[0087] S608, processing the extended image sequence based on the interpolation technology and the video motion compensation technology to generate a compensated image corresponding to each extended image.

[0088] Since the key frame image is only part of the image frames of the initial video stream rather than all the image frames, in order to avoid the video stream from being stuck or having inconsistent sizes when playing, in this application, for each extended image, the interpolation technology is used to convert it into a continuous frame sequence. For each continuous frame sequence, the video motion compensation technology is used to process it to eliminate image distortion and deformation caused by factors such as motion blur and jitter during video playback.

[0089] S609, combining the extended image sequence with the compensated image to obtain a target video stream generated by the combination.

[0090] The extended image sequence and the compensated image are arranged in the order of timestamps and combined to obtain the target video stream generated after the combination.

[0091] S610, displaying target images included in the target video stream on a target display device in sequence.

[0092] The embodiments of the present application avoid the problem of black filling area or picture going beyond the device screen caused by the proportional scaling method, and also avoid the problem of picture aspect ratio changing and picture deformation caused by non-proportional scaling. Instead, it directly responds to the target display device, realizes full-screen expansion of the initial video stream, adapts to the size of the target display device, does not change the video resolution, and does not scale the original video ratio, so that videos shot by any device can be played in the full screen of any device, and can provide users with the best video viewing experience.

[0093] Figure 8 is a schematic diagram of an exemplary implementation of an image expansion method shown in the present application, which is applicable to the scenario of performing image expansion on the entire video stream, such as Figure 8 As shown, the image expansion method includes the following steps:

[0094] S801: Acquire an initial video stream to be expanded.

[0095] In addition to directly expanding the image, it is common that the video size often does not match the screen size of the playback device, so the video stream also needs to be expanded. In this application, the video stream to be expanded is used as the initial video stream.

[0096] S802, obtaining all video frame images contained in the initial video stream, and taking each video frame image as an initial image.

[0097] S803: Acquire an initial image to be expanded and image parameters of the initial image.

[0098] S804, obtaining screen parameters of the target display device.

[0099] S805: Determine an extended area corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge ratio.

[0100] S806: Expand the initial image based on the expansion area to generate an expanded image of the initial image.

[0101] Regarding the specific implementation of steps S803 to S806, reference may be made to the specific introduction of the relevant parts in the above embodiments, which will not be elaborated here.

[0102] S807, generating a target video stream based on the extended images according to the order of the initial images in the initial video stream.

[0103] According to the frame number, timestamp and other data of each initial image, the order of each initial image in the initial video stream is determined, and the extended images are sorted according to the order of the initial images in the initial video stream to generate a target video stream.

[0104] S808, displaying the target images included in the target video stream in sequence on the target display device.

[0105] The embodiments of the present application solve the problem of mismatch between the video size and the screen size of the playback device. It will not change the resolution of the video, nor will it scale the original video ratio, so that videos shot by any device can be played in the full screen of any device, providing users with the best video viewing experience.

[0106] Fig. 9 is a schematic diagram of an image expansion device shown in the present application, such as Fig. 9 As shown, the image expansion device 900 includes:

[0107] The first acquisition module 901 is used to acquire an initial image to be expanded and image parameters of the initial image.

[0108] The second acquisition module 902 is used to acquire screen parameters of a target display device.

[0109] The area determination module 903 is used to determine the extended area corresponding to the initial image based on the image parameters, the screen parameters and the preset image edge ratio.

[0110] The image expansion module 904 is configured to expand the initial image based on the expansion area to generate an expanded image of the initial image.

[0111] Furthermore, the image extension module 904 is also used to: obtain image features of the initial image, and generate extended text of the initial image in combination with the extended area; input the extended text and the initial image into the image extension model to obtain the extended image output by the image extension model.

[0112] Furthermore, the image expansion device 900 also includes: a judgment module 905, which is used to judge whether the extended image requires super-resolution processing based on image parameters and screen parameters; a first processing module 906, which is used to perform super-resolution processing on the extended image in response to judging that the extended image requires super-resolution processing, so as to obtain a target image obtained after super-resolution processing; a second processing module 907, which is used to use the extended image as the target image in response to judging that the extended image does not require super-resolution processing; and a first display module 908, which is used to display the target image on a target display device.

[0113] Furthermore, the first acquisition module 901 is also used to: acquire an initial video stream to be extended; perform key frame extraction on the initial video stream, and use the extracted key frame images as initial images; and the image extension module 904 is also used to: sort the extended images according to the order of the initial images in the initial video stream, and acquire an extended image sequence generated after sorting; process the extended image sequence based on interpolation technology and video motion compensation technology, and generate a compensated image corresponding to each extended image; combine the extended image sequence with the compensated image, and acquire the target video stream generated after the combination.

[0114] Furthermore, the first acquisition module 901 is also used to: obtain the initial video stream to be expanded; obtain all video frame images contained in the initial video stream, and use each video frame image as the initial image; and the image expansion module 904 is also used to: generate a target video stream based on the expanded image in the order of the initial images in the initial video stream.

[0115] Furthermore, the image expansion device 900 further includes: a second display module 909, which is used to display the target images contained in the target video stream on the target display device in sequence.

[0116] Fig.10 is a block diagram of an electronic device 1000 according to an exemplary embodiment.

[0117] like Fig.10 As shown, the electronic device 1000 includes:

[0118] The memory 1001 and the processor 1002, a bus 1003 connecting different components (including the memory 1001 and the processor 1002), the memory 1001 stores a computer program, and when the processor 1002 executes the program, the image expansion method of the embodiment of the present disclosure is implemented.

[0119] Bus 1003 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0120] The electronic device 1000 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 1000, including volatile and non-volatile media, removable and non-removable media.

[0121] The memory 1001 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1004 and / or cache memory 1005. The electronic device 1000 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1006 may be used to read and write non-removable, non-volatile magnetic media ( Fig.10 not shown, usually called a "hard drive"). Although Fig.10 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 1003 via one or more data medium interfaces. The memory 1001 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0122] A program / utility 1008 having a set (at least one) of program modules 1007 may be stored, for example, in the memory 1001. Such program modules 1007 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 1007 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0123] The electronic device 1000 may also communicate with one or more external devices 1009 (e.g., keyboard, pointing device, display 1010, etc.), one or more devices that enable a user to interact with the electronic device 1000, and / or any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 1011. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1012. Fig.10 As shown, the network adapter 1012 communicates with other modules of the electronic device 1000 via the bus 1003. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0124] The processor 1002 executes various functional applications and data processing by running the programs stored in the memory 1001 .

[0125] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the image expansion method of the embodiment of the present disclosure, and will not be repeated here.

[0126] In order to implement the above embodiment, the embodiment of the present application further proposes a non-transient computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to implement the image expansion method shown in the above embodiment. Optionally, the non-transient computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0127] In order to implement the above embodiment, the embodiment of the present application further proposes a computer program product, including a computer program, which implements the image expansion method shown in the above embodiment when executed by a processor.

[0128] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for expanding an image, characterized in that: include: Acquire an initial image to be expanded and image parameters of the initial image; Get the screen parameters of the target display device; Determining an extended area corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge ratio; The initial image is image-extended based on the extended region to generate an extended image of the initial image.

2. The method according to claim 1, characterized in that The performing image expansion on the initial image based on the expansion area to generate an extended image of the initial image includes: Acquire image features of the initial image, and generate extended text of the initial image in combination with the extended area; The extended text and the initial image are input into an image extension model, and the extended image output by the image extension model is obtained.

3. The method according to claim 1 or 2, characterized in that: After the image expansion is performed on the initial image based on the expansion area to generate an extended image of the initial image, the method further includes: Determining whether the extended image requires super-resolution processing based on the image parameters and the screen parameters; In response to determining that the extended image needs to be subjected to super-resolution processing, super-resolution processing is performed on the extended image to obtain a target image obtained after the super-resolution processing; In response to determining that the extended image does not need to be subjected to super-resolution processing, taking the extended image as a target image; The target image is displayed on the target display device.

4. The method according to claim 1, characterized in that: The step of obtaining an initial image to be expanded includes: Obtaining the initial video stream to be expanded; Extracting key frames from the initial video stream, and using the extracted key frame images as the initial images; and, After performing image expansion on the initial image based on the expansion area to generate an expanded image of the initial image, the method further includes: Sorting the extended images according to the order of the initial images in the initial video stream, and obtaining a sequence of extended images generated after the ordering; Processing the extended image sequence based on the interpolation technology and the video motion compensation technology to generate a compensated image corresponding to each of the extended images; The extended image sequence is combined with the compensated image to obtain a target video stream generated by the combination.

5. The method according to claim 1, characterized in that The step of obtaining an initial image to be expanded includes: Obtaining the initial video stream to be expanded; Acquire all video frame images contained in the initial video stream, and use each of the video frame images as the initial image; and, After performing image expansion on the initial image based on the expansion area to generate an expanded image of the initial image, the method further includes: A target video stream is generated based on the extended images according to the order of the initial images in the initial video stream.

6. The method according to claim 4 or 5, characterized in that: The method further comprises: The target images contained in the target video stream are displayed in sequence on the target display device.

7. An image expansion device, characterized in that: include: A first acquisition module, used for acquiring an initial image to be expanded and image parameters of the initial image; A second acquisition module is used to acquire screen parameters of a target display device; A region determination module, configured to determine an extended region corresponding to the initial image based on the image parameters, the screen parameters and a preset image edge ratio; The image expansion module is used to expand the initial image based on the expansion area to generate an expanded image of the initial image.

8. The device according to claim 7, characterized in that The image expansion module is further used for: Acquire image features of the initial image, and generate extended text of the initial image in combination with the extended area; The extended text and the initial image are input into an image extension model, and the extended image output by the image extension model is obtained.

9. The device according to claim 7 or 8, characterized in that The device further comprises: A judging module, configured to judge whether the extended image needs super-resolution processing based on the image parameters and the screen parameters; A first processing module, configured to, in response to determining that the extended image needs to be subjected to super-resolution processing, perform super-resolution processing on the extended image to obtain a target image obtained after the super-resolution processing; A second processing module, configured to use the extended image as a target image in response to determining that the extended image does not need to be subjected to super-resolution processing; The first display module is used to display the target image on the target display device.

10. The device according to claim 7, characterized in that The first acquisition module is further used for: Obtaining the initial video stream to be expanded; Extracting key frames from the initial video stream, and using the extracted key frame images as the initial images; and, The image extension module is further used to: sort the extended images according to the order of the initial images in the initial video stream to obtain an extended image sequence generated after sorting; process the extended image sequence based on the interpolation technology and the video motion compensation technology to generate a compensated image corresponding to each of the extended images; The extended image sequence is combined with the compensated image to obtain a target video stream generated by the combination.

11. The device according to claim 7, characterized in that The first acquisition module is further used for: Obtaining the initial video stream to be expanded; Acquire all video frame images contained in the initial video stream, and use each of the video frame images as the initial image; and, The image expansion module is further used to generate a target video stream based on the expanded images according to the order of the initial images in the initial video stream.

12. The device according to claim 10 or 11, characterized in that The device further comprises: The second display module is used to display the target images contained in the target video stream on the target display device in sequence.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps according to any one of claims 1 to 6.

Images