Method for playing back spatial image, and electronic device
By adaptively adjusting the playback strategy of spatial images according to the field of view of the recording and playback devices, the problem of depth distortion of spatial images when played back on different devices is solved, thereby improving the realism and aesthetics.
Patent Information
- Application Number
- PCT/CN2026/090625
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-06
- Filing Date
- 2026-04-15
- Publication Date
- 2026-11-12
AI Technical Summary
There is a significant difference between the field of view of the recording device and the field of view of the playback device, which causes the spatial image to appear flattened or stretched in depth during playback, affecting the sense of realism.
Based on the FOV of the recorded spatial images and the FOV of the playback device, the image playback strategy is adaptively adjusted, and the realism of the images is maintained on different binocular playback devices through projection and cropping techniques.
On different binocular playback devices, this avoids obvious compression or stretching of spatial images in terms of depth, thus improving the realism and aesthetics of spatial images.
Smart Images

Figure CN2026090625_12112026_PF_FP_ABST
Abstract
Description
Methods and electronic devices for playing back spatial images
[0001] This application claims priority to Chinese Patent Application No. 202510579400.0, filed on May 6, 2025, entitled “Method and Electronic Device for Playing Back Spatial Images”, the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of image playback, and more specifically, to a method and electronic device for playing back spatial images. Background Technology
[0003] Spatial images are a media format that has emerged in recent years. They are typically captured simultaneously by two lenses and played back on devices with binocular playback capabilities, such as virtual reality (VR) and augmented reality (AR). During playback, images captured by one lens are displayed on each of the device's left and right screens. By utilizing the parallax of the left and right eyes, the content ultimately appears to the viewer with a strong sense of depth.
[0004] However, there may be a significant difference between the field of view (FOV) of the recording device and the field of view of the playback device, which can easily cause the playback of spatial images to lose its realism. Summary of the Invention
[0005] This application provides a method and electronic device for playing back spatial images, which can play back spatial images according to the recording FOV and the playback FOV used by the playback device. It can ensure that when the spatial image is played back on different binocular playback devices, the played spatial image does not show obvious compression or stretching in depth, so that the played spatial image has a sense of realism.
[0006] In a first aspect, a method for playing back a spatial image is provided. The method is applied to a first device and includes: acquiring a left-eye image and a right-eye image of a first spatial image; acquiring a first image corresponding to the left-eye image based on a first field of view (FOV) and a second FOV, and acquiring a second image corresponding to the right-eye image based on a third FOV and a second FOV, wherein the first FOV is the recording FOV corresponding to the left-eye image, the second FOV is the FOV used by the first device to play back the first spatial image, and the third FOV is the recording FOV corresponding to the right-eye image; displaying the first image on the left screen of the first device, and displaying the second image on the right screen of the first device.
[0007] In this embodiment, the spatial image can be played back according to the recording FOV and the playback FOV used by the playback device. When the spatial image is played back on different binocular playback devices, the played spatial image does not show obvious compression or stretching in depth, so that the played spatial image has a sense of realism.
[0008] In conjunction with the first aspect, in one possible implementation, obtaining the first image corresponding to the left-eye image based on the first FOV and the second FOV includes: projecting the left-eye image onto a first screen corresponding to the second FOV based on the first FOV and the second FOV; and capturing an image on the first screen using a left-eye virtual camera to obtain the first image. Obtaining the second image corresponding to the right-eye image based on the third FOV and the second FOV includes: projecting the right-eye image onto a second screen corresponding to the second FOV based on the third FOV and the second FOV; and capturing an image on the second screen using a right-eye virtual camera to obtain the second image.
[0009] Optionally, the first and second curtains can be the same curtain or two identical curtains.
[0010] In conjunction with the first aspect, in one possible implementation, projecting the left-eye image onto the first screen based on the first FOV and the second FOV includes: when the first FOV is less than or equal to the second FOV, projecting the left-eye image onto the first screen based on the first FOV.
[0011] In this embodiment, when the recording field of view of the image is smaller than the playback field of view of the image, the image can be placed on the playback screen at its original size without performing operations such as shrinking, enlarging or cropping. This can avoid the phenomenon of depth compression or stretching of the playback spatial image, making the playback spatial image have a sense of realism.
[0012] In conjunction with the first aspect, in one possible implementation, projecting the left-eye image onto the first screen based on a first FOV and a second FOV includes: when the first FOV is less than the second FOV, projecting the left-eye image onto the first screen based on a fourth FOV, wherein the fourth FOV is less than or equal to the second FOV and the fourth FOV is greater than the first FOV.
[0013] In this embodiment of the application, when the recording field of view of the image is smaller than the playback field of view of the image, the image can be slightly enlarged for aesthetic reasons, which can make the playback spatial image more realistic while improving the aesthetics of the playback screen.
[0014] In conjunction with the first aspect, in one possible implementation, projecting the left eye image onto the first screen according to a first FOV and a second FOV includes: when the first FOV is greater than the second FOV and the difference between the first FOV and the second FOV is less than a first threshold, reducing the left eye image based on the second FOV; and projecting the reduced left eye image onto the first screen.
[0015] In this embodiment, when the recording field of view of the image is slightly larger than the playback field of view of the image, since the slight reduction has little impact on the realism of the spatial image, the image can be reduced in size so that all the content of the spatial image is within the playback field of view. This allows the user to see more complete content without affecting the realism of the spatial image.
[0016] In conjunction with the first aspect, in one possible implementation, projecting the left-eye image onto the first screen according to a first FOV and a second FOV includes: when the first FOV is greater than the second FOV, cropping the left-eye image based on the second FOV; and projecting the cropped left-eye image onto the first screen.
[0017] In this embodiment of the application, when the recording field of view of the image is greater than the playback field of view of the image, the area of the image that exceeds the playback field of view can be cropped, which can avoid the stretching in depth caused by the shrinking of the spatial image, so that the playback spatial image has a sense of realism.
[0018] In addition, while making the replayed spatial images realistic, it also supports viewing spatial images on devices with a small field of view.
[0019] In conjunction with the first aspect, in one possible implementation, projecting the left-eye image onto the first screen according to a first FOV and a second FOV includes: cropping the left-eye image based on a fifth FOV when the first FOV is greater than or equal to the second FOV, wherein the fifth FOV is less than the second FOV; or cropping the left-eye image based on the fifth FOV when the first FOV is less than the second FOV and the first FOV is greater than the fifth FOV; and projecting the cropped left-eye image onto the first screen.
[0020] In this embodiment, when the recording field of view of the image is greater than the playback field of view of the image, the image can be cropped based on a field of view that is smaller than the playback field of view. This can make the playback spatial image realistic while keeping the content of the spatial image from filling the playback field of view. For example, it can be manifested as having a white area around the playback field of view, which can improve the aesthetics of the spatial image playback.
[0021] In conjunction with the first aspect, in one possible implementation, cropping the left eye image includes: determining a cropping region of the left eye image based on its content, the cropping region including key content in the left eye image; and cropping the left eye image based on the cropping region.
[0022] In this embodiment of the application, when cropping is required when placing an image on a playback screen, the cropping area of the image can be determined according to the content of the image, so that the key content in the image falls into the cropping area. This can make the spatial image in the playback have a sense of realism while avoiding the loss of key image content during the playback process.
[0023] In conjunction with the first aspect, in one possible implementation, cropping the left eye image includes: determining the cropping region of the left eye image based on the user's interaction; and cropping the left eye image based on the cropping region.
[0024] In this embodiment of the application, when cropping is required when placing an image on a playback screen, the cropping area of the image can be determined based on the user's interactive operation. This can improve the interactive experience between the user and the playback device while making the spatial image in the playback more realistic.
[0025] In conjunction with the first aspect, in one possible implementation, after the left eye image is projected onto the first screen, the first screen includes a first blank area, which is the area of the unfilled image on the first screen. The method further includes: beautifying the first blank area, which includes pattern filling or special effects filling.
[0026] In this embodiment of the application, when the image placed on the screen does not fill the screen, the area on the screen where no image content is placed can be beautified, which can improve the aesthetics of the playback screen while making the spatial image of the playback more realistic.
[0027] In conjunction with the first aspect, in one possible implementation, the first blank area is beautified by: beautifying the first blank area according to the content of the left eye image.
[0028] In this embodiment of the application, when the image placed on the screen does not fill the screen, the area on the screen where no image content is placed can be beautified according to the content of the image, which can make the spatial image played back more realistic and further improve the aesthetics of the playback screen.
[0029] In conjunction with the first aspect, in one possible implementation, the second FOV is the optical FOV of the first device; or the second FOV is smaller than the optical FOV of the first device.
[0030] In this embodiment, the FOV used by the playback device to play back the image can be the optical FOV of the playback device, or any FOV smaller than the optical FOV of the playback device. This allows for flexible adjustment of the playback FOV of the spatial image while ensuring that the spatial image has a sense of realism during playback, thereby improving the user's experience of watching spatial videos using the playback device.
[0031] In conjunction with the first aspect, in one possible implementation, before obtaining the first image corresponding to the left-eye image based on the first FOV and the second FOV, the method further includes: determining the first FOV based on the content of the left-eye image.
[0032] In this embodiment of the application, when key information of an image is lost, the key information can be recovered based on the content of the image. In this way, even if the recorded FOV of the image is unknown, the spatial image played back can still have a sense of realism.
[0033] In conjunction with the first aspect, in one possible implementation, projecting the right-eye image onto the second screen based on the third FOV and the second FOV includes: when the third FOV is less than or equal to the second FOV, projecting the right-eye image onto the second screen based on the third FOV.
[0034] In this embodiment, when the recording field of view of the image is smaller than the playback field of view of the image, the image can be placed on the playback screen at its original size without performing operations such as shrinking, enlarging or cropping. This can avoid the phenomenon of depth compression or stretching of the playback spatial image, making the playback spatial image have a sense of realism.
[0035] In conjunction with the first aspect, in one possible implementation, projecting the right-eye image onto the second screen based on the third FOV and the second FOV includes: when the third FOV is smaller than the second FOV, projecting the right-eye image onto the second screen based on the fourth FOV, wherein the fourth FOV is less than or equal to the second FOV and the fourth FOV is greater than the third FOV.
[0036] In this embodiment of the application, when the recording field of view of the image is smaller than the playback field of view of the image, the image can be slightly enlarged for aesthetic reasons, which can make the playback spatial image more realistic while improving the aesthetics of the playback screen.
[0037] In conjunction with the first aspect, in one possible implementation, projecting the right eye image onto the second screen based on the third FOV and the second FOV includes: when the third FOV is greater than the second FOV and the difference between the third FOV and the second FOV is less than a first threshold, reducing the right eye image based on the second FOV; and projecting the reduced right eye image onto the second screen.
[0038] In this embodiment, when the recording field of view of the image is slightly larger than the playback field of view of the image, since the slight reduction has little impact on the realism of the spatial image, the image can be reduced in size so that all the content of the spatial image is within the playback field of view. This allows the user to see more complete content without affecting the realism of the spatial image.
[0039] In conjunction with the first aspect, in one possible implementation, projecting the right-eye image onto the second screen based on the third FOV and the second FOV includes: cropping the right-eye image based on the second FOV when the third FOV is larger than the second FOV; and projecting the cropped right-eye image onto the second screen.
[0040] In this embodiment of the application, when the recording field of view of the image is greater than the playback field of view of the image, the area of the image that exceeds the playback field of view can be cropped, which can avoid the stretching in depth caused by the shrinking of the spatial image, so that the playback spatial image has a sense of realism.
[0041] In addition, while making the replayed spatial images realistic, it also supports viewing spatial images on devices with a small field of view.
[0042] In conjunction with the first aspect, in one possible implementation, projecting the right-eye image onto the second screen based on the third FOV and the second FOV includes: cropping the right-eye image based on the fifth FOV when the third FOV is greater than or equal to the second FOV, wherein the fifth FOV is less than the second FOV; or cropping the right-eye image based on the fifth FOV when the third FOV is less than the second FOV and the third FOV is greater than the fifth FOV; and projecting the cropped right-eye image onto the second screen.
[0043] Optionally, the cropping regions of the left and right images can be the same or not exactly the same.
[0044] In this embodiment, when the recording field of view of the image is greater than the playback field of view of the image, the image can be cropped based on a field of view that is smaller than the playback field of view. This can make the playback spatial image realistic while keeping the content of the spatial image from filling the playback field of view. For example, it can be manifested as having a white area around the playback field of view, which can improve the aesthetics of the spatial image playback.
[0045] In conjunction with the first aspect, in one possible implementation, cropping the right eye image includes: determining a cropping region of the right eye image based on its content, the cropping region including key content in the right eye image; and cropping the right eye image based on the cropping region.
[0046] In this embodiment of the application, when cropping is required when placing an image on a playback screen, the cropping area of the image can be determined according to the content of the image, so that the key content in the image falls into the cropping area. This can make the spatial image in the playback have a sense of realism while avoiding the loss of key image content during the playback process.
[0047] In conjunction with the first aspect, in one possible implementation, cropping the right eye image includes: determining the cropping region of the right eye image based on the user's interaction; and cropping the right eye image based on the cropping region.
[0048] In this embodiment of the application, when cropping is required when placing an image on a playback screen, the cropping area of the image can be determined based on the user's interactive operation. This can improve the interactive experience between the user and the playback device while making the spatial image in the playback more realistic.
[0049] In conjunction with the first aspect, in one possible implementation, after the right eye image is projected onto the second screen, the second screen includes a first blank area, which is the area of the unfilled image on the second screen. The method further includes: beautifying the first blank area, which includes pattern filling or special effects filling.
[0050] In this embodiment of the application, when the image placed on the screen does not fill the screen, the area on the screen where no image content is placed can be beautified, which can improve the aesthetics of the playback screen while making the spatial image of the playback more realistic.
[0051] In conjunction with the first aspect, in one possible implementation, the first blank area is beautified by: beautifying the first blank area according to the content of the right eye image.
[0052] In this embodiment of the application, when the image placed on the screen does not fill the screen, the area on the screen where no image content is placed can be beautified according to the content of the image, which can make the spatial image played back more realistic and further improve the aesthetics of the playback screen.
[0053] In conjunction with the first aspect, in one possible implementation, the second FOV is the optical FOV of the first device; or the second FOV is smaller than the optical FOV of the first device.
[0054] In this embodiment, the FOV used by the playback device to play back the image can be the optical FOV of the playback device, or any FOV smaller than the optical FOV of the playback device. This allows for flexible adjustment of the playback FOV of the spatial image while ensuring that the spatial image has a sense of realism during playback, thereby improving the user's experience of watching spatial videos using the playback device.
[0055] In conjunction with the first aspect, in one possible implementation, before obtaining the second image corresponding to the right eye image based on the third FOV and the second FOV, the method further includes: determining the third FOV based on the content of the right eye image.
[0056] In this embodiment of the application, when key information of an image is lost, the key information can be recovered based on the content of the image. In this way, even if the recorded FOV of the image is unknown, the spatial image played back can still have a sense of realism.
[0057] In conjunction with the first aspect, in one possible implementation, the first device includes a virtual reality (VR) device or an augmented reality (AR) device.
[0058] In conjunction with the first aspect, in one possible implementation, the spatial image includes a spatial photograph or a spatial video.
[0059] In a second aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the first aspect or any possible implementation thereof.
[0060] Thirdly, a computer-readable storage medium is provided, which stores a computer program or instructions that, when executed, implement the method described in the first aspect or any possible implementation thereof.
[0061] Fourthly, a chip is provided, wherein instructions are stored that, when executed on a device, cause the chip to perform the methods described in the first aspect or any possible implementation thereof.
[0062] Fifthly, a computer program product is provided, which stores a computer program or instructions that, when executed, implement the method in the first aspect or any possible implementation of the first aspect. Attached Figure Description
[0063] Figure 1 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application;
[0064] Figure 2 is a software structure block diagram of the electronic device provided in an embodiment of this application;
[0065] Figure 3 is a software structure block diagram of another electronic device provided in an embodiment of this application;
[0066] Figure 4 is a schematic diagram of a recording FOV and a playback FOV for interpreting spatial images provided in an embodiment of this application;
[0067] Figure 5 is a schematic flowchart of a method for playing back spatial images provided in an embodiment of this application;
[0068] Figure 6 is a schematic diagram of placing the left eye image / right eye image on the playback screen according to an embodiment of this application;
[0069] Figure 7 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0070] Figure 8 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0071] Figure 9 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0072] Figure 10 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in the embodiment of this application;
[0073] Figure 11 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0074] Figure 12 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0075] Figure 13 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0076] Figure 14 is a schematic diagram of another placement of the left eye image / right eye image on the playback screen provided in an embodiment of this application;
[0077] Figure 15 is a schematic flowchart of a method for playing back spatial images provided in an embodiment of this application;
[0078] Figure 16 is a schematic flowchart of another method for playing back spatial images provided in an embodiment of this application;
[0079] Figure 17 is a schematic diagram of the functional modules of a device for playing back spatial images provided in an embodiment of this application. Detailed Implementation
[0080] The technical solutions of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments.
[0081] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "plural" or "multiple" refers to two or more than two.
[0082] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.
[0083] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.
[0084] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0085] Spatial images are a media format that has emerged in recent years. They are typically captured simultaneously by two lenses and played back on devices with binocular playback capabilities, such as virtual reality (VR) and augmented reality (AR). During playback, images captured by one lens are displayed on each of the device's left and right screens. By utilizing the parallax of the left and right eyes, the content ultimately appears to the viewer with a strong sense of depth.
[0086] However, there may be a significant difference between the field of view (FOV) of the recording device and the field of view of the playback device.
[0087] Currently, during the playback of 2D images, the recorded images are scaled down or enlarged based on the playback FOV of the playback device.
[0088] This solution is only suitable for 2D image playback scenarios. If applied to spatial image playback scenarios, it will cause the spatial image to appear compressed or stretched in depth, severely affecting the realism of the spatial image. Specifically, if the spatial image is magnified according to the playback FOV of the playback device, the spatial image is easily compressed in depth; if the spatial image is reduced according to the playback FOV of the playback device, the spatial image is easily stretched in depth.
[0089] In view of this, embodiments of this application provide a method and electronic device for playing back spatial images. Through this method and electronic device, the playback strategy of the spatial image is adaptively adjusted according to the recording FOV and the playback FOV used by the playback device. This enables the spatial image to be played back on different binocular playback devices without significant compression or stretching in depth, thus making the played-back spatial image realistic.
[0090] Furthermore, it enables the spatial images to be played back in binoculars on the same binocular playback device using different playback FOVs without significant compression or stretching in depth, thus making the played spatial images appear realistic.
[0091] The method provided in this application can be applied to electronic devices with binocular playback capabilities, such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other electronic devices. This application does not impose any restrictions on the specific type of electronic device.
[0092] For example, Figure 1 shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0093] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0094] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0095] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.
[0096] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0097] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0098] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.
[0099] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.
[0100] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.
[0101] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.
[0102] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.
[0103] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.
[0104] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.
[0105] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.
[0106] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0107] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.
[0108] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.
[0109] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.
[0110] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.
[0111] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.
[0112] The SIM card interface 195 is used to connect the SIM card.
[0113] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.
[0114] The electronic device provided in this application embodiment can run an operating system (OS). This operating system can be various operating systems currently used in the industry, such as HarmonyOS based on OpenHarmony; or other operating systems such as Android™, iOS mobile operating system; it can also be various open-source operating systems or their derivatives, such as Linux OS, and other embedded operating systems; or it can be a future new operating system, such as an AI operating system based on artificial intelligence. An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. The operating system occupies a pivotal position in electronic devices, connecting to the physical devices at the hardware layer below and providing a runtime environment for application software above.
[0115] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer is a set of software, or frameworks, that provides various services to application developers, such as databases, multimedia, and graphics, or capabilities like distributed scheduling and system expansion. For example, the middleware layer can also be broadly divided into a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the core capabilities of the system, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer can include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.
[0116] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. These operating systems share commonalities but also have their own unique characteristics. Different operating systems affect user experience, application ecosystem, and system performance. The basic functions implemented by the electronic device provided in this application can be achieved using a general-purpose operating system or a dedicated operating system.
[0117] To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, Figure 2 exemplarily illustrates the architecture of HarmonyOS. Those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as Android. TMImplementation under the operating system.
[0118] As shown in Figure 2, the software architecture of an electronic device can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. The layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem granularity in different device deployment scenarios. Each subsystem can also be tailored, added, or combined at the functional granularity.
[0119] (1) Kernel layer
[0120] The Kernel Abstraction Layer (KAL) provides basic kernel capabilities to upper layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management.
[0121] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux kernel, HarmonyOS kernel, LiteOS, etc.
[0122] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.
[0123] (2) System Service Layer
[0124] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:
[0125] The system's basic capability subsystems provide fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. For example, they may include distributed soft bus, distributed data management, distributed task scheduling, and the Ark multi-language runtime. They may also include multi-modal input subsystems, graphics subsystems, security subsystems, and AI subsystems.
[0126] Basic software service subsystems: provide common and general software services; for example, event notification subsystem, telephone service subsystem, multimedia subsystem, etc.
[0127] Enhanced software service subsystem suite: Provides differentiated capability-enhancing software services for different devices; for example, it may include proprietary business subsystems for smart screens, wearable devices, and IoT devices.
[0128] Hardware service subsystem set: provides hardware services; for example, it may include location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, etc.
[0129] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.
[0130] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.
[0131] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, inter-process communication RPC (Remote Procedure Call) and other communication capabilities.
[0132] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages and multiple chip platforms.
[0133] (3) Framework layer
[0134] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. Examples include the ArkUI framework (which provides a complete infrastructure for UI development of system applications, including UI functionalities such as components, layouts, animations, and interactive events, as well as a real-time interface preview tool), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may run different operating systems, and therefore support different APIs.
[0135] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. Examples include: Audio API (audio service), Push API (push service), and Account API (account service).
[0136] (4) Application layer
[0137] Applications can include system apps and extended / third-party apps. System apps can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party apps can include social apps, travel apps, etc.
[0138] Figure 3 is a software structure block diagram of another electronic device 100 provided in an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, Android is used... TM The system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system library layer, and the kernel layer. The application layer can include a series of application packages.
[0139] As shown in Figure 3, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.
[0140] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.
[0141] As shown in Figure 3, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
[0142] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.
[0143] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.
[0144] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.
[0145] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).
[0146] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.
[0147] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.
[0148] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.
[0149] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.
[0150] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.
[0151] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.
[0152] A 2D graphics engine is a graphics engine for 2D drawing.
[0153] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.
[0154] It should be understood that the technical solutions in the embodiments of this application can be used in Android. TM In systems such as iOS and HarmonyOS.
[0155] For example, with reference to Figure 4, the recording field of view (FOV) and playback FOV of spatial images are explained in detail.
[0156] As shown in Figure 4(a), the FOV of a spatial image refers to the angle between the edge of the recorded image and the line connecting the lens.
[0157] It is understandable that each optical instrument corresponds to a field of view (FOV), and the size of the FOV determines the field of view of the optical instrument.
[0158] The playback FOV of a spatial image refers to the FOV used by the playback device when playing back the spatial image. The playback FOV can be an optical FOV or a non-optical FOV. For example, as shown in Figure 4(b), the playback FOV can be the angle between the edge of the displayed image and the line connecting the viewpoint (eye). The playback FOV of the spatial image can be calculated using the diagonal of the displayed image.
[0159] Here, optical FOV refers to the FOV of the optical module of the playback device, which may include, for example, AR or VR devices.
[0160] In some embodiments, the playback FOV (horizontal FOV) of the spatial image can also be calculated using the long side dimension of the displayed image, and the playback FOV (vertical FOV) of the spatial image can also be calculated using the short side dimension of the displayed image.
[0161] For example, Figure 5 shows a schematic flowchart of a method 500 for playing back a spatial image provided in an embodiment of this application.
[0162] As shown in Figure 5, the method 500 includes:
[0163] S501: The playback device determines the distance 1 between the observation point and the playback screen.
[0164] In some embodiments, the point of view may be, for example, the eye of a user viewing a spatial image.
[0165] The distance 1 between the observation point and the playback screen can be determined based on scene information.
[0166] S502: The playback device determines the size 1 of the playback screen based on distance 1 and playback FOV.
[0167] Among them, playback FOV refers to the FOV of the playback device.
[0168] The playback screen can be understood as a virtual screen, and its size is variable.
[0169] S503: The playback device places the left eye image of the spatial image onto the playback screen.
[0170] S504: The playback device places the right eye image of the spatial image onto the playback screen.
[0171] In some embodiments, two lenses can be used to capture spatial images simultaneously, wherein the image captured by the left lens is the left-eye image of the spatial image, and the image captured by the right lens is the right-eye image of the spatial image.
[0172] In some embodiments, a 2D image can be captured using a lens, and the 2D image can be processed by an algorithm to obtain the left-eye image and the right-eye image of the corresponding spatial image.
[0173] In some embodiments, spatial images can be generated using virtual objects such as 3D models instead of being captured by photography to obtain spatial images.
[0174] In some embodiments, spatial images may be acquired not by taking pictures, but by generating left-eye and right-eye images of the spatial image based on user instructions using artificial intelligence.
[0175] In some embodiments, the playback device can place the left / right eye image of the spatial image on the playback screen according to the recording field of view corresponding to the spatial image and the playback field of view used by the playback device.
[0176] In some embodiments, after the playback device determines the size of the playback screen, it first executes S503 and S505, that is, first places the left eye image of the spatial image on the playback screen, and then performs algorithmic processing on the left eye image placed on the playback screen, for example, using a left eye virtual camera to capture the left eye image placed on the playback screen, and further executes S507, that is, sends the image captured by the virtual camera to the left eye screen of the playback device for display; after executing S503 and S505, it further executes S504 and S506, that is, places the right eye image of the spatial image on the playback screen, and then performs algorithmic processing on the right eye image placed on the playback screen, for example, using a right eye virtual camera to capture the right eye image placed on the playback screen, and further executes S508, that is, sends the image captured by the virtual camera to the right eye screen of the playback device for display.
[0177] Among them, a virtual camera refers to a camera that uses software algorithms to simulate shooting and output the captured content, based on the digital content corresponding to the image.
[0178] In some embodiments, after determining the size of the playback screen, the playback device sets two screens with the same size and other parameters, for example, denoted as the left screen and the right screen. The playback device can execute S503 and S504 simultaneously or at different times. For example, the playback device can place the left-eye image of the spatial image on the left screen, and then perform algorithm processing on the left-eye image placed on the left screen, for example, using a left-eye virtual camera to capture the left-eye image placed on the left screen, and sending the image captured by the virtual camera to the left-eye screen of the playback device for display. Simultaneously, the playback device can place the right-eye image of the spatial image on the right screen, and then perform algorithm processing on the right-eye image placed on the right screen, for example, using a right-eye virtual camera to capture the right-eye image placed on the right screen, and sending the image captured by the virtual camera to the right-eye screen of the playback device for display.
[0179] S505: The playback device processes the left eye image placed on the playback screen.
[0180] In some embodiments, the playback device performs algorithmic processing on the left-eye image placed on the playback screen.
[0181] In one example, the playback device performs algorithmic processing on the left-eye image placed on the playback screen, including processing the left-eye image placed on the playback screen using a left-eye virtual camera.
[0182] S506: Process the right eye image placed on the playback screen.
[0183] In some embodiments, the playback device performs algorithmic processing on the right-eye image placed on the playback screen.
[0184] In one example, the playback device performs algorithmic processing on the right-eye image placed on the playback screen, including processing the right-eye image placed on the playback screen using a right-eye virtual camera.
[0185] In some embodiments, the recording baseline of the spatial image needs to be incorporated into the processing of the left / right eye images placed on the playback screen.
[0186] In some implementations, when the recording baseline of the spatial image is unknown, the recording baseline of the spatial image can be recovered based on the content of the left and right eye images corresponding to the spatial image.
[0187] When the spatial image is recorded using two lenses, the recording baseline refers to the distance between the two lenses when recording the spatial image; when the spatial image is generated by an algorithm, the recording baseline refers to the distance between the generated left-eye image and the generated right-eye image.
[0188] S507: Send the processed left eye image to the left eye screen for display.
[0189] S508: Send the processed right eye image to the right eye screen for display.
[0190] It can be understood that the method provided in this application embodiment is applied to the execution process of S503 and S504 above, specifically by placing the left eye image / right eye image of the spatial image on the playback screen based on a matching strategy.
[0191] Hereinafter, with reference to Figures 6 to 14, we will introduce several placement strategies for placing the left / right eye images of spatial images on the playback screen provided by the embodiments of this application.
[0192] In some embodiments, the placement strategy for placing the left-eye image on the playback screen can be determined based on the recording FOV and playback FOV corresponding to the left-eye image; the placement strategy for placing the right-eye image on the playback screen can be determined based on the recording FOV and playback FOV corresponding to the right-eye image.
[0193] For example, Figure 6 shows a schematic diagram of placing the left eye image / right eye image on the playback screen according to an embodiment of this application.
[0194] As shown in Figure 6(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV2.
[0195] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0196] It is understood that the playback FOV used by the playback device in the embodiments of this application does not specifically refer to the optical FOV of the playback device. It can be any FOV that is less than or equal to the optical FOV of the playback device. For example, it can be a preset FOV. For instance, it can be a user-preset FOV according to their needs, or it can be a FOV determined according to the playback scenario. For instance, when a user wants to play back an image in the form of a preview window, the playback FOV can refer to the FOV corresponding to the preview window.
[0197] As shown in Figure 6(b), when the difference between FOV2 and FOV1 is less than a1 and the difference between FOV2 and FOV1 is greater than a2, when placing image 600 on the playback screen, no scaling, scaling or cropping operation is performed on image 600, that is, image 600 is placed on the playback screen according to its original scale.
[0198] Where a1 and a2 are both positive numbers greater than 0, and a1 is greater than a2.
[0199] This strategy can also be understood as follows: when the recording FOV of image 600 is much smaller than the playback FOV, no scaling, enlarging, or cropping operation is performed on image 600; that is, image 600 is placed on the playback screen according to its original size.
[0200] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0201] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0202] In this embodiment of the application, when the recording field of view of the image is much smaller than the playback field of view of the image, the image can be placed on the playback screen at its original size without performing operations such as shrinking, enlarging or cropping. This can avoid the phenomenon of depth compression or stretching of the playback spatial image, making the playback spatial image have a sense of realism.
[0203] For example, Figure 7 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0204] As shown in Figure 7(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV3.
[0205] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0206] As shown in Figure 7(b), when the difference between FOV3 and FOV1 is less than a2, and the difference between playback FOV and recording FOV is greater than or equal to 0, when image 600 is placed on the playback screen, no scaling, scaling or cropping operation is performed on image 600, that is, image 600 is placed on the playback screen according to its original scale.
[0207] Where a2 is a positive number greater than 0.
[0208] This strategy can also be understood as follows: when the recording FOV of image 600 is smaller than the playback FOV, but the recording FOV is similar to the playback FOV, no scaling, enlarging, or cropping operation is performed on image 600, that is, image 600 is placed on the playback screen according to its original size.
[0209] It is understandable that when the recording field of view of an image is slightly smaller than the playback field of view of an image, after placing the image at 600 degrees as in this example, a blank area will appear around the playback screen, for example, a blank area around the edge of the playback screen.
[0210] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0211] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0212] In this embodiment, when the recording field of view of the image is slightly smaller than the playback field of view of the image, the image can be placed on the playback screen at its original size without performing operations such as shrinking, enlarging or cropping. This avoids the phenomenon of depth compression or stretching of the playback spatial image, making the playback spatial image more realistic and improving the aesthetics of the playback screen.
[0213] For example, Figure 8 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0214] As shown in Figure 8(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV4.
[0215] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0216] As shown in Figure 8(b), when FOV4 is greater than FOV1, when image 600 is placed on the playback screen, a magnification operation is performed on image 600 based on FOV1+a3.
[0217] The operation of magnifying image 600 based on FOV1+a3 means magnifying the size of image 600 to be the same as the size of the area on the playback screen corresponding to the field of view FOV1+a3.
[0218] Where a3 is a positive number greater than 0, and a3 is less than or equal to the difference between FOV4 and FOV1.
[0219] This strategy can also be understood as: when the recording FOV of image 600 is smaller than the playback FOV, a slight magnification operation is performed on image 600.
[0220] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0221] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0222] In this embodiment of the application, when the recording field of view of the image is smaller than the playback field of view of the image, the image can be slightly enlarged for aesthetic reasons, which can make the playback spatial image more realistic while improving the aesthetics of the playback screen.
[0223] For example, Figure 9 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0224] As shown in Figure 9(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV5.
[0225] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0226] As shown in Figure 9(b) and Figure 9(c), when FOV5 is less than FOV1, when image 600 is placed on the playback screen, a cropping operation is performed on image 600 based on FOV5. The cropped image is image 700, and the final image placed on the screen is image 700. At this time, image 700 fills the playback FOV, that is, there is no blank area on the screen.
[0227] The cropping operation on image 600 based on FOV5 means cropping the size of image 600 to be the same as the size of the area on the playback screen corresponding to the field of view FOV5. In other words, the size of the cropped area of image 600 is the same as the size of the area on the playback screen corresponding to the field of view FOV5.
[0228] This strategy can also be understood as follows: when the recording FOV of image 600 is greater than the playback FOV, the image 600 is not scaled down; instead, the area of image 600 that exceeds the playback FOV is cropped.
[0229] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0230] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0231] In this embodiment of the application, when the recording field of view of the image is greater than the playback field of view of the image, the area of the image that exceeds the playback field of view can be cropped, which can avoid the stretching in depth caused by the image shrinkage, so that the spatial image of the playback has a sense of realism.
[0232] In addition, while making the replayed spatial images realistic, it also supports viewing spatial images on devices with a small field of view.
[0233] For example, Figure 10 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0234] As shown in Figure 10(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV6.
[0235] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0236] When the difference between FOV1 and FOV6 is greater than a4, when image 600 is placed on the playback screen, a cropping operation is performed on image 600 based on FOV6. The cropped image is image 800, and the final image placed on the screen is image 800. At this time, image 800 fills the playback FOV, that is, there is no blank area on the screen.
[0237] The cropping operation on image 600 based on FOV6 means cropping the size of image 600 to be the same as the size of the area on the playback screen corresponding to the field of view FOV6. In other words, the size of the cropped area of image 600 is the same as the size of the area on the playback screen corresponding to the field of view FOV6.
[0238] Where a4 is a positive number greater than 0.
[0239] As shown in Figure 10(b), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV7.
[0240] When the difference between FOV1 and FOV7 is greater than 0 and the difference between FOV1 and FOV7 is less than or equal to a4, when image 600 is placed on the playback screen, image 600 is scaled down based on FOV7. The scaled-down image is image 900, and the final image placed on the screen is image 900. At this time, image 900 fills the playback FOV, that is, there is no blank area on the screen.
[0241] The operation of reducing the size of image 600 based on FOV7 means reducing the size of image 600 to the same size as the area on the playback screen corresponding to the field of view FOV7.
[0242] This strategy can also be understood as follows: when the recording FOV of image 600 is significantly larger than the playback FOV, no scaling operation is performed on image 600; instead, a cropping operation is performed on the area of image 600 that exceeds the playback FOV. When the recording FOV of image 600 is slightly larger than the playback FOV, since the slight scaling does not significantly affect the realism of the spatial image, a scaling operation can be performed on image 600, and no cropping operation is performed on the area of image 600 that exceeds the playback FOV, allowing users to see more complete content.
[0243] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0244] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0245] In this embodiment of the application, when the recording field of view of the image is slightly larger than the playback field of view of the image, since the slight reduction has little impact on the realism of the spatial image, the image can be reduced in size so that all the content of the image is within the playback field of view. This allows the user to see more complete content without affecting the realism of the spatial image.
[0246] For example, Figure 11 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0247] As shown in Figure 11(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV8.
[0248] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0249] As shown in Figure 11(b), when FOV8 is less than FOV1, when image 600 is placed on the playback screen, a cropping operation can be performed on image 600 based on FOV8-a5. The cropped image is image 1000, and the field of view corresponding to image 1000 is FOV8-a5. The final image placed on the screen is image 1000. At this time, image 1000 does not fill the playback FOV, that is, there is a blank area on the screen.
[0250] The cropping operation on image 600 based on FOV8-a5 means cropping the size of image 600 to be the same as the size of the area on the playback screen corresponding to the field of view FOV8-a5. In other words, the size of the cropped area of image 600 is the same as the size of the area on the playback screen corresponding to the field of view FOV8-a5.
[0251] Where a5 is a positive number greater than 0.
[0252] In some embodiments, a5 can be a constant, such as a constant set by default in the playback device, or a constant determined based on user input.
[0253] In some embodiments, a5 can be a variable, such as a variable that is dynamically adjusted according to information such as the recording FOV, recording baseline, and playback FOV of image 600.
[0254] This strategy can also be understood as follows: when the recording FOV of image 600 is greater than the playback FOV, a cropping operation can be performed on the area of image 600 that exceeds FOV8-a5. The FOV of the cropped image is smaller than the playback FOV, which can ensure that the image placed on the screen does not fill the playback POV.
[0255] In some embodiments, when FOV8 equals FOV1, when placing image 600 on the playback screen, a cropping operation can also be performed on image 600 based on FOV8-a5, and the field of view corresponding to the cropped image is FOV8-a5.
[0256] In some embodiments, when FOV8 is greater than FOV1 and FOV1 is greater than FOV8-a5, when placing image 600 on the playback screen, a cropping operation can also be performed on image 600 based on FOV8-a5, and the field of view corresponding to the cropped image is FOV8-a5.
[0257] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0258] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0259] In this embodiment of the application, when the recording field of view of the image is greater than the playback field of view of the image, the image can be cropped based on a field of view that is smaller than the playback field of view. This can make the playback spatial image realistic while keeping the image content from filling the playback field of view. For example, it can be manifested as having a white area around the playback field of view, which can improve the aesthetics of the spatial image playback.
[0260] For example, Figure 12 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0261] As shown in Figure 12(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV9.
[0262] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0263] As shown in Figure 12(b), the playback FOV used by the playback device to play back the image 600 may not be the optical FOV of the playback device. The playback FOV used may be dynamically adjusted within a range smaller than the optical FOV. For example, it may be the FOV corresponding to the preview window on the playback screen.
[0264] In some embodiments, when FOV9 is greater than FOV1, when placing image 600 on the playback screen based on the FOV corresponding to the preview window on the playback screen, image 600 can be placed directly in the FOV9 area on the screen, or image 600 can be slightly magnified within the range not exceeding FOV9 before being placed in the FOV9 area on the screen.
[0265] In some embodiments, when FOV9 is smaller than FOV1, when the image 600 is placed on the playback screen based on the FOV corresponding to the preview window on the playback screen, the area of the image 600 that exceeds FOV9 can be cropped, and the cropped image can be placed within the FOV9 area on the screen. If FOV9 is slightly smaller than FOV1, the image can be shrunk based on FOV9, and the shrunk image can be placed within the FOV9 area on the screen.
[0266] It should be understood that this example uses the playback FOV of the playback device as the FOV of the preview window, which is smaller than the optical FOV of the playback device. However, this does not limit the definition of playback FOV provided in this application. The playback FOV can also be other FOVs smaller than the optical FOV of the playback device. For example, it can be the FOV of a small window on the screen, the FOV of a full-screen window on the screen, or other FOVs that are dynamically adjusted within a range not greater than the optical FOV. This application does not limit these.
[0267] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0268] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0269] In this embodiment, the FOV used by the playback device to play back the image can be the optical FOV of the playback device, or any FOV smaller than the optical FOV of the playback device. This allows for flexible adjustment of the playback FOV of the spatial image while ensuring that the spatial image has a sense of realism during playback, thereby improving the user's experience of watching spatial videos using the playback device.
[0270] For example, Figure 13 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0271] As shown in Figure 13(a), the recording FOV corresponding to image 600 is FOV1, meaning that image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV10. Since FOV1 is smaller than FOV10, image 600 is placed on a screen with a field of view of FOV10 at a field of view of FOV1. Because FOV1 is smaller than FOV10, i.e., the recording field of view of the image is smaller than the playback field of view of the image, there is a blank area around the perimeter of the field of view FOV10, as shown by area A in Figure 13(a).
[0272] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0273] In some embodiments, the aforementioned blank areas can be beautified through transition processing.
[0274] For example, as shown in Figure 13(b), the blank area can be filled with shadows, such as gradient shadow filling or filling with different colors.
[0275] For example, as shown in Figure 13(c), the blank area can be filled with a pattern, such as a multi-dot pattern, a wood grain pattern, a hazy pattern, a floral pattern, etc.
[0276] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0277] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0278] It is understood that the playback device can perform transitional beautification on the blank area based on the content of the played-out image, or it can perform transitional beautification on the blank area based on a random pattern or a preset pattern. This application does not limit the method of transitional beautification.
[0279] In this embodiment of the application, when the image placed on the screen does not fill the screen, the area on the screen where no image content is placed can be beautified, which can improve the aesthetics of the playback screen while making the spatial image of the playback more realistic.
[0280] For example, Figure 14 shows another schematic diagram of placing the left / right eye image on the playback screen according to an embodiment of this application.
[0281] As shown in Figure 14(a), the recording FOV corresponding to image 600 is FOV1, that is, image 600 was captured using a lens with a field of view of FOV1; the playback FOV used by the playback device to play back image 600 is FOV11, where FOV11 is smaller than FOV1. When image 600 is placed on the playback screen, image 600 can be cropped based on FOV11, that is, the area of image 600 that exceeds FOV11 is cropped, so that the FOV corresponding to image 600 is reduced to FOV11.
[0282] Cropping image 600 based on FOV11 means cropping the size of image 600 to be the same as the size of the area on the playback screen corresponding to the field of view FOV10. In other words, the size of the cropped area of image 600 is the same as the size of the area on the playback screen corresponding to the field of view FOV11.
[0283] Optionally, image 600 can be either the left-eye image or the right-eye image of a spatial image.
[0284] As shown in Figure 14(a), when cropping the region of image 600 that exceeds FOV11, the cropped region of image 600 can be centered relative to the region of image 600.
[0285] As shown in Figures 14(b) and 14(c), in order to ensure that the cropped image includes the key content of image 600, the cropping area of image 600 can be adjusted. For example, if the cropping area is centered relative to the area of image 600, and the cropping area does not include the complete human head, the cropping area of image 600 can be moved to the upper left so that the cropping area includes the complete human head, resulting in the cropped image 1100. The cropped image 1100 is then placed on the playback screen, and this image 1100 includes the complete human head.
[0286] In some embodiments, the playback device can dynamically adjust the cropping area of the playback image based on the content of the playback image.
[0287] Specifically, the playback device can perform feature analysis on the playback image to determine the key content in the playback image, and then determine the cropping area of the playback image based on the key content in the playback image, so that the key content in the playback image falls into the cropping area of the playback image and does not fall into the cropped area of the playback image.
[0288] In some embodiments, the playback device may dynamically adjust the cropped area of the playback image in response to user interaction.
[0289] The user's interactive operations may include, for example, dragging and dropping the cropping area; or the user's head movement, such as the user looking up to the right and the cropping area of the playback image moving to the right and upward; or other user gestures, etc. This application does not limit the user's interactive operations.
[0290] In some embodiments, considering the issue of left and right eye fusion, when cropping the left and right eye images of the same spatial image separately, the cropping areas of the left eye image and the right eye image may not be exactly the same.
[0291] In some embodiments, when the recorded FOV of image 600 is unknown (e.g., parameters are missing), the image content of image 600 can be parsed, and the recorded FOV of image 600 can be recovered from the image content of image 600. In this way, even when the recorded FOV of an image is unknown, the spatial image being played back can still have a sense of realism.
[0292] In some implementations, the recorded FOV of image 600 can be recovered from the image content of image 600 using methods such as model inference.
[0293] In this embodiment of the application, when cropping is required when placing an image on a playback screen, the cropping area of the image can be dynamically determined so that the key content in the image falls into the cropping area. This can make the spatial image in the playback feel realistic while avoiding the loss of key image content during the playback process.
[0294] In some embodiments, the spatial images described in this application may include spatial photographs or spatial videos.
[0295] With the spatial image playback solution provided in this application, the same spatial photo or video can be played back and displayed on devices with different FOVs at a completely consistent or roughly similar scale, and there will be no abnormalities such as compression or stretching in depth.
[0296] It is understood that the embodiments shown in Figures 5 to 14 above can be implemented individually or in any combination thereof.
[0297] For example, FIG15 shows a schematic flowchart of a method 1500 for playing back spatial images provided in an embodiment of the present application.
[0298] As shown in Figure 15, the method 1500 includes:
[0299] S1501: Obtain the left eye image and the right eye image of the first spatial image.
[0300] S1502: Obtain the first image corresponding to the left eye image of the first spatial image based on the first FOV and the second FOV.
[0301] Wherein, the first FOV is the recording FOV corresponding to the left eye image of the first spatial image, and the second FOV is the FOV used by the first device to play back the first spatial image.
[0302] The first image includes the left eye image of the first spatial image.
[0303] S1503: Obtain the second image corresponding to the right eye image of the first spatial image based on the third FOV and the second FOV.
[0304] The third FOV is the recorded FOV corresponding to the right eye image of the first spatial image.
[0305] The second image includes the right eye image of the first spatial image.
[0306] S1504: Display the first image on the left screen of the first device, and display the second image on the right screen of the first device.
[0307] In this embodiment, the spatial image can be played back according to the recording FOV and the playback FOV used by the playback device. When the spatial image is played back on different binocular playback devices, the played spatial image does not show obvious compression or stretching in depth, so that the played spatial image has a sense of realism.
[0308] For example, FIG16 shows a schematic flowchart of another method 1600 for playing back spatial images provided in an embodiment of the present application.
[0309] As shown in Figure 16, the method 1600 includes:
[0310] S1601: Obtain the left eye image and the right eye image of the first spatial image.
[0311] S1602: Project the left-eye image of the first spatial image onto the first screen corresponding to the second FOV according to the first FOV and the second FOV.
[0312] Wherein, the first FOV is the recording FOV corresponding to the left eye image of the first spatial image, and the second FOV is the FOV used by the first device to play back the first spatial image.
[0313] S1603: Use the left-eye virtual camera to capture an image on the first screen to obtain the first image corresponding to the left-eye image of the first spatial image.
[0314] S1604: Project the right-eye image of the first spatial image onto the second screen corresponding to the second FOV based on the third FOV and the second FOV.
[0315] The third FOV is the recorded FOV corresponding to the right eye image of the first spatial image.
[0316] S1605: Use the right-eye virtual camera to capture an image on the second screen to obtain a second image corresponding to the right-eye image of the first spatial image.
[0317] S1606: Display the first image on the left screen of the first device, and display the second image on the right screen of the first device.
[0318] For example, FIG17 shows a schematic diagram of the functional modules of a device 1700 for playing back spatial images provided in an embodiment of the present application.
[0319] As shown in Figure 17, the device 1700 includes an acquisition module 1710, a processing module 1720, and a display module 1730, wherein:
[0320] The acquisition module 1710 is used to acquire the left eye image and the right eye image of the first spatial image.
[0321] The processing module 1720 is used to obtain the first image corresponding to the left eye image of the first spatial image based on the first FOV and the second FOV.
[0322] Wherein, the first FOV is the recording FOV corresponding to the left eye image of the first spatial image, and the second FOV is the FOV used by the first device to play back the first spatial image.
[0323] The first image includes the left eye image of the first spatial image.
[0324] In some embodiments, the processing module 1720 is specifically configured to project the left-eye image of the first spatial image onto a first screen corresponding to the second FOV according to the first FOV and the second FOV; and use a left-eye virtual camera to capture an image on the first screen to obtain a first image corresponding to the left-eye image of the first spatial image.
[0325] The processing module 1720 is also used to obtain the second image corresponding to the left eye image of the first spatial image based on the third FOV and the second FOV.
[0326] The third FOV is the recorded FOV corresponding to the right eye image of the first spatial image.
[0327] The second image includes the right eye image of the first spatial image.
[0328] In some embodiments, the processing module 1720 is specifically configured to project the right-eye image of the first spatial image onto the second screen corresponding to the second FOV, based on the third FOV and the second FOV.
[0329] Display module 1730 is used to display a first image on the left screen of the first device and to display a second image on the right screen of the first device.
[0330] One or more modules or units described herein can be implemented in software, hardware, or a combination of both. When any of the above modules or units are implemented in software, the software exists as computer program instructions and is stored in memory. A processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor can be built into a SoC (System-on-a-Chip) or an application-specific integrated circuit (ASIC), or it can be a separate semiconductor chip. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.
[0331] When the modules or units described herein are implemented in hardware, the hardware may be any one or any combination of a CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, application-specific digital circuit, hardware accelerator, or non-integrated discrete device, which may run the necessary software or perform the above method flow independently of software.
[0332] When the modules or units described herein are implemented using software, they can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0333] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0334] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0335] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0336] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0337] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0338] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0339] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for playing back spatial images, characterized in that, The method is applied to a first device, and the method includes: Obtain the left and right eye images of the first spatial image; The first image corresponding to the left eye image is obtained based on the first field of view (FOV) and the second FOV, and the second image corresponding to the right eye image is obtained based on the third FOV and the second FOV. The first FOV is the recording FOV corresponding to the left eye image, the second FOV is the FOV used by the first device to play back the first spatial image, and the third FOV is the recording FOV corresponding to the right eye image. The first image is displayed on the left screen of the first device, and the second image is displayed on the right screen of the first device.
2. The method according to claim 1, characterized in that, The step of obtaining the first image corresponding to the left eye image based on the first FOV and the second FOV includes: The left-eye image is projected onto the first screen corresponding to the second FOV based on the first FOV and the second FOV; Use the left-eye virtual camera to capture an image on the first screen to obtain the first image; The step of obtaining the second image corresponding to the right eye image based on the third FOV and the second FOV includes: The right-eye image is projected onto the second screen corresponding to the second FOV based on the third FOV and the second FOV; Use the right-eye virtual camera to capture an image on the second screen to obtain the second image.
3. The method according to claim 2, characterized in that, The step of projecting the left-eye image onto the first screen based on the first FOV and the second FOV includes: When the first FOV is less than or equal to the second FOV, the left eye image is projected onto the first screen based on the first FOV.
4. The method according to claim 2 or 3, characterized in that, The step of projecting the left-eye image onto the first screen according to the first FOV and the second FOV includes: When the first FOV is less than the second FOV, the left eye image is projected onto the first screen based on the fourth FOV, wherein the fourth FOV is less than or equal to the second FOV and the fourth FOV is greater than the first FOV.
5. The method according to any one of claims 2 to 4, characterized in that, The step of projecting the left-eye image onto the first screen according to the first FOV and the second FOV includes: When the first FOV is greater than the second FOV, and the difference between the first FOV and the second FOV is less than a first threshold, the left eye image is reduced based on the second FOV; The reduced left-eye image is projected onto the first screen.
6. The method according to any one of claims 2 to 5, characterized in that, The step of projecting the left-eye image onto the first screen according to the first FOV and the second FOV includes: When the first FOV is greater than the second FOV, the left eye image is cropped based on the second FOV; The cropped left-eye image is projected onto the first screen.
7. The method according to any one of claims 2 to 6, characterized in that, The step of projecting the left-eye image onto the first screen according to the first FOV and the second FOV includes: When the first FOV is greater than or equal to the second FOV, the left eye image is cropped based on the fifth FOV, wherein the fifth FOV is less than the second FOV; or When the first FOV is smaller than the second FOV and the first FOV is larger than the fifth FOV, the left eye image is cropped based on the fifth FOV; The cropped left-eye image is projected onto the first screen.
8. The method according to claim 6 or 7, characterized in that, The cropping of the left eye image includes: The cropping region of the left eye image is determined based on its content, and the cropping region includes key content from the left eye image. The left eye image is cropped according to the cropping region.
9. The method according to any one of claims 6 to 8, characterized in that, The cropping of the left eye image includes: The cropping area of the left eye image is determined based on the user's interactive operations; The left eye image is cropped according to the cropping region.
10. The method according to any one of claims 3, 4, and 7, characterized in that, After the left-eye image is projected onto the first screen, the first screen includes a first blank area, which is the area of the unfilled image on the first screen. The method further includes: The first blank area is beautified, and the beautification process includes pattern filling or special effects filling.
11. The method according to claim 10, characterized in that, The enhancement process for the first blank area includes: The first blank area is beautified based on the content of the left eye image.
12. The method according to any one of claims 1 to 11, characterized in that, The second FOV is the optical FOV of the first device; or the second FOV is smaller than the optical FOV of the first device.
13. The method according to any one of claims 1 to 12, characterized in that, Before obtaining the first image corresponding to the left-eye image based on the first FOV and the second FOV, the method further includes: The first FOV is determined based on the content of the left-eye image.
14. The method according to any one of claims 1 to 13, characterized in that, The first device includes a virtual reality (VR) device or an augmented reality (AR) device.
15. The method according to any one of claims 1 to 14, characterized in that, The spatial images include spatial photographs or spatial videos.
16. An electronic device, characterized in that, include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 13.
17. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 13.
18. A chip, characterized in that, The chip stores instructions that, when executed, implement the method as described in any one of claims 1 to 13.
19. A computer program product, characterized in that, The computer program product stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 13.
