Panoramic picture optimization method, device, storage medium and electronic device

Abstract

The present invention discloses a method and device for optimizing a panoramic picture, the method comprising: obtaining a partial panoramic picture, wherein the partial panoramic picture includes a partial foreground object picture and a partial background object picture; setting the lens shrinkage of the panoramic camera, and performing a zoom operation on the partial foreground object picture based on the lens shrinkage to obtain a reduced partial foreground object picture; obtaining at least one auxiliary panoramic picture, and obtaining an auxiliary partial background object picture that does not include a foreground object based on the auxiliary panoramic picture; performing a zoom operation on the auxiliary partial background object picture based on the lens shrinkage to obtain a reduced auxiliary partial background object picture; superimposing the reduced partial foreground object picture with the reduced auxiliary partial background object picture to generate an optimized partial panoramic picture. By using the embodiments of the present invention, the interval area without picture content between the foreground and the background can be eliminated, thereby improving the user experience.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular to a method, a device, a storage medium and an electronic device for optimizing a panoramic picture. Background Art

[0002] Existing large panoramic cameras have multiple lenses in different directions. Since the generation of panoramic 3D images based on 3D panoramic cameras requires the use of captured images from multiple cameras at the same time, the generated panoramic 3D images need to meet the requirements of the standard viewing circle of the human body, which is generally a circle of about 65mm in diameter formed based on the average spacing of human eyes (about 65mm). If the above requirements are not met, the 3D images viewed by users may be distorted.

[0003] With the development of technology, users have higher and higher requirements for the image acquisition of 3D panoramic cameras. The size of 3D panoramic camera systems is getting larger and larger. The viewing circle size corresponding to 3D panoramic cameras is generally larger than the above-mentioned standard viewing circle, which will cause distortion when users watch the 3D images of the above-mentioned 3D panoramic cameras.

[0004] At present, the existing panoramic cameras are large in size, resulting in a large field of view of the camera, which does not match the field of view of the human head. Therefore, the field of view of the camera needs to be shrunk. The content captured by the lens includes the foreground and the background. When the field of view is shrunk, there will be an interval area with no picture content between the foreground and the background, which affects the user experience. This is a problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for optimizing a panoramic picture to solve the deficiencies in the prior art. The method and device can eliminate the interval area without picture content between the foreground and the background, thereby improving the user experience.

[0006] An embodiment of the present application provides a method for optimizing a panoramic picture, wherein the panoramic picture is obtained by photographing a panoramic camera having multiple lenses, and the optimization method includes:

[0007] Acquire a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0008] Setting a zoom-in amount of the panoramic camera lens, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens, to obtain a zoomed-out local foreground object image;

[0009] Acquire at least one auxiliary panoramic picture, and acquire an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0010] Performing a zooming operation on the auxiliary local background object picture based on the zooming amount of the lens to obtain a zoomed-out auxiliary local background object picture;

[0011] The reduced local foreground object image is superimposed with the reduced auxiliary local background object image to generate an optimized local panoramic image.

[0012] Optionally, the step of acquiring at least one auxiliary panoramic picture includes:

[0013] The background object picture of the current picture frame of the panoramic video is obtained at a set time interval, and the original background object picture is updated in sequence using a plurality of the background object pictures to obtain the auxiliary panoramic picture.

[0014] Optionally, the step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture is:

[0015] Based on the relative positional relationship between the local foreground object picture and the local background object picture, the reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

[0016] Optionally, the step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture is:

[0017] Acquire a first boundary area between the local foreground object picture and the local background object picture;

[0018] Selecting some picture features in the first boundary area as first boundary features;

[0019] Based on the first boundary feature, the reduced local foreground object picture and the reduced auxiliary local background object picture are superimposed to generate an optimized local panoramic picture.

[0020] Optionally, in the step of selecting part of the picture features in the first boundary area as the boundary features, a plurality of first boundary features are selected;

[0021] The step of superimposing the reduced local foreground object picture and the reduced auxiliary local background object picture based on the first boundary feature to generate an optimized local panoramic picture includes:

[0022] The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features.

[0023] Optionally, the step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture further includes:

[0024] Acquire adjacent local panoramic pictures taken by adjacent lenses, wherein the adjacent local panoramic pictures include adjacent local foreground object pictures and adjacent local background object pictures;

[0025] Acquire a second boundary region between the adjacent local foreground object pictures and the adjacent local background object pictures;

[0026] Selecting some picture features in the second junction area as second boundary features;

[0027] Based on the second boundary feature and the first boundary feature, the reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

[0028] Optionally, in the step of selecting some of the picture features in the second junction area as the second boundary features, a plurality of second boundary features are selected;

[0029] The step of superimposing the reduced local foreground object picture and the reduced auxiliary local background object picture based on the second boundary feature and the first boundary feature to generate an optimized local panoramic picture includes:

[0030] The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features and second boundary features.

[0031] An embodiment of the present application provides a device for optimizing a panoramic picture, wherein the panoramic picture is obtained by photographing a panoramic camera having multiple lenses, and the device comprises:

[0032] A first acquisition module is used to acquire a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0033] A first zoom-out module, used for setting a zoom-in amount of the lens of the panoramic camera, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens to obtain a zoomed-out local foreground object image;

[0034] A second acquisition module is used to acquire at least one auxiliary panoramic picture, and acquire an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0035] A second zoom-out module, configured to zoom out the auxiliary local background object picture based on the lens zoom-in amount to obtain a zoomed-out auxiliary local background object picture;

[0036] The superposition module is used to superimpose the reduced local foreground object image with the reduced auxiliary local background object image to generate an optimized local panoramic image.

[0037] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.

[0038] Yet another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the methods described above.

[0039] Compared with the prior art, the present invention provides a method for optimizing a panoramic picture, wherein the panoramic picture is obtained by shooting a panoramic camera with multiple lenses, by obtaining a partial panoramic picture, wherein the partial panoramic picture includes a partial foreground object picture and a partial background object picture; setting the lens indentation of the panoramic camera, and performing a zoom operation on the partial foreground object picture based on the lens indentation to obtain a reduced partial foreground object picture; obtaining at least one auxiliary panoramic picture, and obtaining an auxiliary partial background object picture excluding a foreground object based on the auxiliary panoramic picture; performing a zoom operation on the auxiliary partial background object picture based on the lens indentation to obtain a reduced auxiliary partial background object picture; superimposing the reduced partial foreground object picture with the reduced auxiliary partial background object picture to generate an optimized partial panoramic picture. Through the lens indentation, the partial foreground object picture and the complete auxiliary partial background object picture excluding the foreground object are proportionally reduced, and then superimposed to generate an optimized partial panoramic picture, thereby eliminating the interval area without picture content between the foreground and the background, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A hardware structure block diagram of a computer terminal of a panoramic picture optimization system provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a flow chart of a method for optimizing a panoramic image provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a panoramic camera structure provided by an embodiment of the present invention;

[0043] Figure 4A schematic diagram of a field of view of a panoramic camera provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of the positions of a panoramic camera lens, foreground objects and background objects provided by an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the structure of a panoramic picture optimization device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, but should not be construed as limiting the present invention.

[0047] The embodiment of the present invention firstly provides a method for optimizing a panoramic picture, which can be applied to electronic devices, such as computer terminals and the like.

[0048] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal of a panoramic picture optimization method provided by an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0049] The memory 104 can be used to store software programs and modules of application software, such as program instructions / modules corresponding to the method for optimizing the panoramic picture in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0051] See also Figure 2 , Figure 2 A schematic flow chart of a method for optimizing a panoramic picture provided by an embodiment of the present invention, wherein the panoramic picture is obtained by photographing a panoramic camera with multiple lenses, and may include the following steps:

[0052] S201, acquiring a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0053] Existing large-scale 3D panoramic cameras have multiple lenses in different directions. Figure 3 The viewing circle size of a 3D panoramic camera is generally larger than the standard viewing circle, such as Figure 4 As shown, the 3D panoramic display screen of the 3D panoramic camera will be distorted when viewed by the user. Based on this, the embodiment of the present invention needs to shrink the lens circle (i.e. shrink the lens circle of the panoramic camera to the standard viewing circle), and expand the display screen, i.e. stretch the display screen, to meet the user's viewing experience of the background display screen.

[0054] like Figure 5 As shown, L1 and L1' are the distances between the lens and the foreground object of the panoramic picture, and L2 and L2' are the distances between the lens and the background object of the panoramic picture. When the lens moves backward, the absolute values ​​of the changes of L1 and L2, L1'-L1 and L2'-L2, are the same, and (L1'-L1) / L1 is greater than (L2'-L2) / L2. Therefore, when the lens moves backward, the change (reduction) of the foreground object is greater than the change (reduction) of the background object, resulting in a gap without picture content between the foreground object and the background object.

[0055] Among them, the partial panoramic picture refers to the partial picture in the entire panoramic picture that can be seen by the human eye at the same time. The panoramic picture taken by the panoramic camera is divided into basically fixed background objects (such as street scenes or blue sky and white clouds) and generally active foreground objects (such as people in front of the camera or moving objects, etc.).

[0056] S202, setting a zoom-in amount of the panoramic camera lens, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens, to obtain a zoomed-out local foreground object image;

[0057] The amount of lens shrinkage can be determined by the user according to actual needs to avoid distortion. Shrinking the image refers to reducing the content displayed on the image, while also expanding the display range of the image.

[0058] S203, acquiring at least one auxiliary panoramic picture, and acquiring an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0059] Specifically, a time interval may be set to obtain the background object picture of the current picture frame of the panoramic video, and a plurality of the background object pictures may be used in sequence to update the original background object picture to obtain the auxiliary panoramic picture.

[0060] Among them, the auxiliary panoramic picture can be a panoramic picture of the same content shot before. In the real video picture frame, the background picture will still change, such as the movement of the sun, the movement of the clouds, or the movement of the clock hand, but the background picture changes slowly (relative to the moving foreground picture). Since the background picture will also change slowly, in order to ensure the accuracy of the synthesized picture, the previous background picture can be updated with the latest background picture. Specifically, the preset time interval can be determined by the user according to actual conditions, and the original background object picture is updated in chronological order. What is updated is the background object picture of the entire panoramic video, not the local background object picture. The original background object picture can be the earliest background object picture shot that does not include the foreground object.

[0061] Exemplarily, assume that 10 frames of background object images are acquired. The image with the earliest shooting time is used as the original background object image, and the original background object image is updated with the second earliest shooting time image to obtain an updated background object image. Then, the updated background object image is updated again with the third earliest shooting time image to obtain a background object image that is updated again. This process is repeated until the background object image after the last update is updated for the last time with the image with the latest shooting time, thereby obtaining an auxiliary local background object image. Each background object image acquired here removes the corresponding foreground object image, but since the foreground object is always moving in the panoramic image, multiple frames of background object images in the panoramic image are superimposed to obtain a complete background object image of the panoramic image.

[0062] S204, performing a zoom-out operation on the auxiliary local background object picture based on the zoom-in amount of the lens to obtain a zoomed-out auxiliary local background object picture;

[0063] Specifically, the auxiliary local background object image is reduced by the same lens zoom-in amount to ensure that during subsequent superposition, the reduced local foreground object image and the reduced auxiliary local background object image are scaled in equal proportion, wherein the greater the distance between the image and the lens, the greater the zoom amount of the image.

[0064] S205 , superimposing the reduced local foreground object image and the reduced auxiliary local background object image to generate an optimized local panoramic image.

[0065] Specifically, based on the relative position relationship between the local foreground object picture and the local background object picture, the reduced local foreground object picture and the reduced auxiliary local background object picture may be superimposed to generate an optimized local panoramic picture.

[0066] Since the background image is collected without the foreground, the separately collected foreground image and the separately collected background image need to be superimposed to obtain the reduced panoramic image. Among them, the position of the reduced current foreground object image in the panoramic image corresponds to the position of the unreduced foreground object image in the panoramic image, so that an optimized partial panoramic image without intervals without image content can be generated.

[0067] In one implementation, a first boundary area between the local foreground object image and the local background object image can be obtained; some image features in the first boundary area are selected as first boundary features; based on the first boundary features, the reduced local foreground object image and the reduced auxiliary local background object image are superimposed to generate an optimized local panoramic image.

[0068] Among them, multiple first boundary features can be selected, and the reduced local foreground object picture and the reduced auxiliary local background object picture are superimposed, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features.

[0069] Exemplarily, the boundary area is the intersection area of ​​the foreground object hand and the background lake surface, and the boundary feature is the key picture feature in the boundary area, as long as the user feels it is important. For example, the intersection point of the user's hand and the lake surface can be the boundary feature point, that is, when the foreground picture and the background picture are superimposed after being reduced, it is necessary to ensure that the reduced user's hand (foreground picture) is also in contact with the reduced lake surface (background picture).

[0070] Furthermore, adjacent partial panoramic images taken by adjacent lenses can be obtained, wherein the adjacent partial panoramic images include adjacent partial foreground object images and adjacent partial background object images; a second boundary region of the adjacent partial foreground object images and the adjacent partial background object images is obtained; some image features in the second boundary region are selected as second boundary features; based on the second boundary features and the first boundary features, the reduced partial foreground object images are superimposed with the reduced auxiliary partial background object images to generate an optimized partial panoramic image. The second boundary features are, for example, the intersection of the foreground person's head and the background white clouds, the intersection of the foreground car and the background trees, and the like.

[0071] Among them, multiple second boundary features can be selected to superimpose the reduced local foreground object picture and the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features and second boundary features.

[0072] Since the panoramic camera includes multiple lenses, after selecting the first boundary feature based on the content of the current lens, the second boundary feature can also be selected based on the content captured by the adjacent lenses because the adjacent lenses have overlapping fields of view. The adjacent partial panoramic images can provide more boundary features to prevent the first boundary features from being insufficient. With sufficient boundary features, the accuracy of the relative position can be guaranteed and the superposition is more accurate.

[0073] It can be seen that by acquiring a partial panoramic picture, wherein the partial panoramic picture includes a partial foreground object picture and a partial background object picture; setting the lens indentation of the panoramic camera, and performing a zoom operation on the partial foreground object picture based on the lens indentation, a zoomed-out partial foreground object picture is obtained; acquiring at least one auxiliary panoramic picture, and acquiring an auxiliary partial background object picture excluding the foreground object based on the auxiliary panoramic picture; performing a zoom operation on the auxiliary partial background object picture based on the lens indentation, and obtaining a zoomed-out auxiliary partial background object picture; superimposing the zoomed-out partial foreground object picture with the zoomed-out auxiliary partial background object picture to generate an optimized partial panoramic picture. Through the lens indentation, the partial foreground object picture and the complete auxiliary partial background object picture excluding the foreground object are proportionally reduced, and then superimposed to generate an optimized partial panoramic picture, thereby eliminating the interval area without picture content between the foreground and the background, and improving the user experience.

[0074] See also Figure 6 , Figure 6 A schematic diagram of a panoramic image optimization device provided by an embodiment of the present invention, and Figure 2Corresponding to the process shown, the panoramic picture is obtained by taking a panoramic camera with multiple lenses, and the device includes:

[0075] A first acquisition module 601 is used to acquire a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0076] A first zoom-out module 602 is used to set the zoom-in amount of the panoramic camera lens, and zoom out the local foreground object image based on the zoom-in amount of the lens to obtain a zoomed-out local foreground object image;

[0077] A second acquisition module 603 is used to acquire at least one auxiliary panoramic picture, and acquire an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0078] A second zoom-out module 604 is configured to zoom out the auxiliary local background object image based on the zoom-in amount of the lens to obtain a zoomed-out auxiliary local background object image;

[0079] The superposition module 605 is used to superimpose the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

[0080] Specifically, the second acquisition module is specifically used to:

[0081] The background object picture of the current picture frame of the panoramic video is obtained at a set time interval, and the original background object picture is updated in sequence using a plurality of the background object pictures to obtain the auxiliary panoramic picture.

[0082] Specifically, the superposition module includes:

[0083] The superposition unit is used to superimpose the reduced local foreground object picture with the reduced auxiliary local background object picture based on the relative position relationship between the local foreground object picture and the local background object picture to generate an optimized local panoramic picture.

[0084] Specifically, the superposition unit includes:

[0085] An acquisition subunit, used for acquiring a first boundary area between the local foreground object picture and the local background object picture;

[0086] A selecting subunit, configured to select a portion of the picture features in the first boundary area as a first boundary feature;

[0087] The superposition subunit is used to superimpose the reduced local foreground object picture and the reduced auxiliary local background object picture based on the first boundary feature to generate an optimized local panoramic picture.

[0088] Specifically, the selecting subunit selects a plurality of first boundary features; the superimposing subunit is specifically used to:

[0089] The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features.

[0090] Specifically, the superposition subunit is further used for:

[0091] Acquire adjacent local panoramic pictures taken by adjacent lenses, wherein the adjacent local panoramic pictures include adjacent local foreground object pictures and adjacent local background object pictures;

[0092] Acquire a second boundary region between the adjacent local foreground object pictures and the adjacent local background object pictures;

[0093] Selecting some picture features in the second junction area as second boundary features;

[0094] Based on the second boundary feature and the first boundary feature, the reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

[0095] Specifically, a plurality of second boundary features are selected in the superposition subunit; the superposition subunit is specifically used for:

[0096] The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features and second boundary features.

[0097] It can be seen that by acquiring a partial panoramic picture, wherein the partial panoramic picture includes a partial foreground object picture and a partial background object picture; setting the lens indentation of the panoramic camera, and performing a zoom operation on the partial foreground object picture based on the lens indentation, a zoomed-out partial foreground object picture is obtained; acquiring at least one auxiliary panoramic picture, and acquiring an auxiliary partial background object picture excluding the foreground object based on the auxiliary panoramic picture; performing a zoom operation on the auxiliary partial background object picture based on the lens indentation, and obtaining a zoomed-out auxiliary partial background object picture; superimposing the zoomed-out partial foreground object picture with the zoomed-out auxiliary partial background object picture to generate an optimized partial panoramic picture. Through the lens indentation, the partial foreground object picture and the complete auxiliary partial background object picture excluding the foreground object are proportionally reduced, and then superimposed to generate an optimized partial panoramic picture, thereby eliminating the interval area without picture content between the foreground and the background, and improving the user experience.

[0098] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0099] Specifically, in this embodiment, the above storage medium may be configured to store a computer program for performing the following steps:

[0100] S201, acquiring a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0101] S202, setting a zoom-in amount of the panoramic camera lens, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens, to obtain a zoomed-out local foreground object image;

[0102] S203, acquiring at least one auxiliary panoramic picture, and acquiring an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0103] S204, performing a zoom-out operation on the auxiliary local background object picture based on the zoom-in amount of the lens to obtain a zoomed-out auxiliary local background object picture;

[0104] S205 , superimposing the reduced local foreground object image and the reduced auxiliary local background object image to generate an optimized local panoramic image.

[0105] Specifically, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0106] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0107] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0108] Specifically, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0109] S201, acquiring a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture;

[0110] S202, setting a zoom-in amount of the panoramic camera lens, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens, to obtain a zoomed-out local foreground object image;

[0111] S203, acquiring at least one auxiliary panoramic picture, and acquiring an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture;

[0112] S204, performing a zoom-out operation on the auxiliary local background object picture based on the zoom-in amount of the lens to obtain a zoomed-out auxiliary local background object picture;

[0113] S205 , superimposing the reduced local foreground object image and the reduced auxiliary local background object image to generate an optimized local panoramic image.

[0114] Specifically, the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0115] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0116] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0117] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0118] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0121] The embodiments of the present invention are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for optimizing a panoramic picture, wherein the panoramic picture is obtained by taking a panoramic camera with multiple lenses. It is characterized in that The optimization method comprises: Acquire a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture; Setting a zoom-in amount of the panoramic camera lens, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens, to obtain a zoomed-out local foreground object image; Acquire at least one auxiliary panoramic picture, and acquire an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture; Performing a zooming operation on the auxiliary local background object picture based on the zooming amount of the lens to obtain a zoomed-out auxiliary local background object picture; Superimposing the reduced local foreground object image with the reduced auxiliary local background object image to generate an optimized local panoramic image; The step of acquiring at least one auxiliary panoramic picture comprises: Acquire the background object picture of the current picture frame of the panoramic video at a set time interval, and sequentially use a plurality of the background object pictures to update the original background object picture to obtain the auxiliary panoramic picture; wherein the original background object picture is the earliest captured background object picture that does not include the foreground object; The step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture is: Acquire a first boundary area between the local foreground object picture and the local background object picture; Selecting some picture features in the first boundary area as first boundary features; Based on the first boundary feature, the reduced local foreground object picture and the reduced auxiliary local background object picture are superimposed to generate an optimized local panoramic picture.

2. The optimization method according to claim 1, It is characterized in that The step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture is: Based on the relative positional relationship between the local foreground object picture and the local background object picture, the reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

3. The optimization method according to claim 1, It is characterized in that In the step of selecting part of the picture features in the first boundary area as the boundary features, a plurality of first boundary features are selected; The step of superimposing the reduced local foreground object picture and the reduced auxiliary local background object picture based on the first boundary feature to generate an optimized local panoramic picture includes: The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features.

4. The optimization method according to claim 1, It is characterized in that The step of superimposing the reduced local foreground object picture with the reduced auxiliary local background object picture to generate an optimized local panoramic picture also includes: Acquire adjacent local panoramic pictures taken by adjacent lenses, wherein the adjacent local panoramic pictures include adjacent local foreground object pictures and adjacent local background object pictures; Acquire a second boundary region between the adjacent local foreground object pictures and the adjacent local background object pictures; Selecting some picture features in the second boundary area as second boundary features; Based on the second boundary feature and the first boundary feature, the reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture to generate an optimized local panoramic picture.

5. The optimization method according to claim 4, It is characterized in that In the step of selecting part of the picture features in the second junction area as the second boundary features, a plurality of second boundary features are selected; The step of superimposing the reduced local foreground object picture and the reduced auxiliary local background object picture based on the second boundary feature and the first boundary feature to generate an optimized local panoramic picture includes: The reduced local foreground object picture is superimposed with the reduced auxiliary local background object picture, so that the superimposed local panoramic picture and the local panoramic picture have the most first boundary features and second boundary features.

6. A device for optimizing a panoramic picture, wherein the panoramic picture is obtained by taking a panoramic camera with multiple lenses. It is characterized in that The optimization device comprises: A first acquisition module is used to acquire a local panoramic picture, wherein the local panoramic picture includes a local foreground object picture and a local background object picture; A first zoom-out module, used for setting a zoom-in amount of the lens of the panoramic camera, and performing a zoom-out operation on the local foreground object image based on the zoom-in amount of the lens to obtain a zoomed-out local foreground object image; A second acquisition module is used to acquire at least one auxiliary panoramic picture, and acquire an auxiliary local background object picture excluding foreground objects based on the auxiliary panoramic picture; A second zoom-out module, configured to zoom out the auxiliary local background object picture based on the lens zoom-in amount to obtain a zoomed-out auxiliary local background object picture; A superposition module, used for superimposing the reduced local foreground object image with the reduced auxiliary local background object image to generate an optimized local panoramic image; The second acquisition module is further used to acquire a background object picture of the current picture frame of the panoramic video at a set time interval, and sequentially use a plurality of the background object pictures to update the original background object picture to obtain the auxiliary panoramic picture; wherein the original background object picture is the earliest captured background object picture that does not include the foreground object; The superposition module is specifically used to obtain a first boundary area between the local foreground object image and the local background object image; select some image features in the first boundary area as a first boundary feature; based on the first boundary feature, superimpose the reduced local foreground object image and the reduced auxiliary local background object image to generate an optimized local panoramic image.

7. A storage medium, It is characterized in that The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

8. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images