Coding method, electronic device, communication system, and storage medium

Abstract

This application provides an encoding / decoding method, an electronic device, a communication system, and a storage medium. The encoding method includes: upon receiving a shooting command from a second device, activating the first camera and the second camera; encoding a first panoramic image to obtain a first compressed image, wherein the first panoramic image is any image in the first panoramic image stream; obtaining a first difference image based on the first panoramic image and a first close-up image, wherein the first close-up image is an image synchronously acquired with the first panoramic image in the first close-up image stream; encoding the first difference image to obtain a second compressed image; and sending the first compressed image and the second compressed image to the second device. Because the close-up image is a close-up of a local area on the panoramic image, the data size of the difference image is very small, and the image jitter problem caused by concurrent I-frames will not occur.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an encoding / decoding method, electronic device, communication system, and storage medium. Background Technology

[0002] When a local device needs to obtain multiple image streams from a remote device, it can control the remote device to open multiple cameras via the network. The remote device can send the image streams captured by each camera to the local device. Each image stream sent by the remote device to the local device has a large amount of data, but network bandwidth is limited. Therefore, the remote device needs to compress each image stream.

[0003] Since each camera captures images synchronously, when the remote device compresses each image stream, the I-frames of the compressed stream often occur simultaneously. The large data volume of the I-frame will cause a sudden increase in the amount of data transmitted over the network, causing the display screen of the local device to jitter and affecting the user experience. Summary of the Invention

[0004] This application provides an encoding / decoding method, electronic device, communication system, and storage medium to solve the image jitter problem caused by concurrent I-frames.

[0005] In a first aspect, embodiments of this application provide a communication system, including: a first device and a second device. The first device includes: a first camera and a second camera. The first camera is used to capture a first panoramic image stream, and the second camera is used to capture a first close-up image stream. Each close-up image in the first close-up image stream is a close-up of a local area on a synchronously captured panoramic image. The first device is used to: upon receiving a shooting command sent by the second device, activate the first camera and the second camera; encode the first panoramic image to obtain a first compressed image, where the first panoramic image is any image in the first panoramic image stream; obtain a first difference image based on the first panoramic image and the first close-up image, where the first close-up image is an image synchronously captured with the first panoramic image in the first close-up image stream; encode the first difference image to obtain a second compressed image; and send the first compressed image and the second compressed image to the second device. The second device is used to: decode the first compressed image to obtain a second panoramic image; decode the second compressed image to obtain a second difference image; and obtain a second close-up image based on the second panoramic image and the second difference image.

[0006] In this embodiment of the application, the first device is also referred to as a remote device, and the second device is also referred to as a local device. In this embodiment of the application, the first camera is also referred to as a panoramic camera, and the second camera is also referred to as a close-up camera.

[0007] In one possible implementation, the first device is specifically used to: determine a first local region corresponding to the first close-up image on the first panoramic image; crop the first local region from the first panoramic image to obtain a first local image; adjust the size of the first local image to be the same as the size of the first close-up image to obtain a first magnified image; and determine the first difference image based on the first magnified image and the first close-up image.

[0008] In one possible implementation, the first device is specifically used to: determine the first local area based on the relative positions of the first camera and the second camera, the zoom level of the first camera, and the zoom level of the second camera.

[0009] In one possible implementation, the first device is specifically used to: determine whether the relative position has changed; if the relative position has changed, then use an image recognition algorithm to determine the first local region; if the relative position has not changed, then determine whether the zoom ratio of the first camera and the zoom ratio of the second camera have changed; if the zoom ratio of the first camera and / or the zoom ratio of the second camera has changed, then determine the first local region based on the current zoom ratio of the first camera and the current zoom ratio of the second camera; if the zoom ratio of the first camera and the zoom ratio of the second camera have not changed, then determine the first local region based on a first region, wherein the first region is the local region corresponding to the previous frame of the first close-up image on the previous frame of the first panoramic image.

[0010] In one possible implementation, the first device is specifically used to: determine the first local region based on the center point of the first panoramic image, the size of the first close-up image, the current zoom level of the first camera, and the current zoom level of the second camera.

[0011] In one possible implementation, the first device is specifically used to: determine the first local area based on the first region, the current zoom level of the first camera, and the current zoom level of the second camera.

[0012] In one possible implementation, the first device is specifically used to: determine the difference in color values ​​of each pixel based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image; and obtain the first difference image based on the difference in color values ​​of each pixel.

[0013] In one possible implementation, the second device is specifically used to: determine a second local region corresponding to the second close-up image on the second panoramic image; crop the second local region from the second panoramic image to obtain a second local image; adjust the size of the second local image to be the same as the size of the second difference image to obtain a second magnified image; and determine the second close-up image based on the second magnified image and the second difference image.

[0014] In one possible implementation, the second device is further configured to: receive information about the first local region sent by the first device; specifically, the second device is configured to: determine the second local region based on the information about the first local region.

[0015] In one possible implementation, the second device is further configured to: receive the current zoom level of the first camera and the current zoom level of the second camera sent by the first device; specifically, the second device is configured to: determine the second local region based on the current zoom level of the first camera and the current zoom level of the second camera.

[0016] In one possible implementation, the second device is specifically used to: determine the second local region based on the center point of the second panoramic image, the size of the second difference image, the current zoom level of the first camera, and the current zoom level of the second camera.

[0017] In one possible implementation, the second device is specifically used to: determine the second local region based on the second region, the current zoom level of the first camera, and the current zoom level of the second camera, wherein the second region is the local region corresponding to the previous frame of the second close-up image on the previous frame of the second panoramic image.

[0018] In one possible implementation, the first device is specifically used to: determine the difference / sum of the color values ​​of each pixel based on the color values ​​of each pixel in the second difference image and the color values ​​of each pixel in the second magnified image; and obtain the second close-up image based on the difference / sum of the color values ​​of each pixel.

[0019] Secondly, embodiments of this application provide an encoding method applied to a first device, the first device including: a first camera and a second camera, the first camera being used to capture a first panoramic image stream, and the second camera being used to capture a first close-up image stream, wherein each close-up image in the first close-up image stream is a close-up of a local area on a synchronously captured panoramic image; the method includes: after receiving a shooting command sent by a second device, activating the first camera and the second camera; encoding the first panoramic image to obtain a first compressed image, the first panoramic image being any image in the first panoramic image stream; obtaining a first difference image based on the first panoramic image and the first close-up image, the first close-up image being an image in the first close-up image stream that was synchronously captured with the first panoramic image; encoding the first difference image to obtain a second compressed image; and sending the first compressed image and the second compressed image to the second device.

[0020] In one possible implementation, obtaining the first difference image based on the first panoramic image and the first close-up image includes: determining a first local region corresponding to the first close-up image on the first panoramic image; cropping the first local region from the first panoramic image to obtain a first local image; adjusting the size of the first local image to be the same as the size of the first close-up image to obtain a first magnified image; and determining the first difference image based on the first magnified image and the first close-up image.

[0021] In one possible implementation, determining the first local region corresponding to the first close-up image on the first panoramic image includes: determining the first local region based on the relative positions of the first camera and the second camera, the zoom level of the first camera, and the zoom level of the second camera.

[0022] In one possible implementation, determining the first local region based on the relative positions of the first camera and the second camera, the zoom level of the first camera, and the zoom level of the second camera includes: determining whether the relative positions have changed; if the relative positions have changed, determining the first local region using an image recognition algorithm; if the relative positions have not changed, determining whether the zoom levels of the first camera and the second camera have changed; if the zoom levels of the first camera and / or the second camera have changed, determining the first local region based on the current zoom levels of the first camera and the second camera; if the zoom levels of both the first camera and the second camera have not changed, determining the first local region based on a first region, wherein the first region is the local region corresponding to the previous frame of the first close-up image on the previous frame of the first panoramic image.

[0023] In one possible implementation, determining the first local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: determining the first local region based on the center point of the first panoramic image, the size of the first close-up image, the current zoom level of the first camera, and the current zoom level of the second camera.

[0024] In one possible implementation, determining the first local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: determining the first local region based on the first region, the current zoom level of the first camera, and the current zoom level of the second camera.

[0025] In one possible implementation, determining the first difference image based on the first magnified image and the first close-up image includes: determining the difference in color values ​​of each pixel based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image; and obtaining the first difference image based on the difference in color values ​​of each pixel.

[0026] Thirdly, embodiments of this application provide a decoding method, including: decoding the first compressed image to obtain a second panoramic image; decoding the second compressed image to obtain a second difference image; and obtaining a second close-up image based on the second panoramic image and the second difference image.

[0027] In one possible implementation, obtaining the second close-up image based on the second panoramic image and the second difference image includes: determining a second local region corresponding to the second close-up image on the second panoramic image; cropping the second local region from the second panoramic image to obtain a second local image; adjusting the size of the second local image to be the same as the size of the second difference image to obtain a second magnified image; and determining the second close-up image based on the second magnified image and the second difference image.

[0028] In one possible implementation, the method further includes: receiving information about the first local region sent by the first device; determining the second local region corresponding to the second close-up image on the second panoramic image includes: determining the second local region based on the information about the first local region.

[0029] In one possible implementation, the method further includes: receiving the current zoom level of the first camera and the current zoom level of the second camera sent by the first device; determining the second local region corresponding to the second close-up image on the second panoramic image includes: determining the second local region based on the current zoom level of the first camera and the current zoom level of the second camera.

[0030] In one possible implementation, determining the second local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: determining the second local region based on the center point of the second panoramic image, the size of the second difference image, the current zoom level of the first camera, and the current zoom level of the second camera.

[0031] In one possible implementation, determining the second local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: determining the second local region based on the second region, the current zoom level of the first camera, and the current zoom level of the second camera, wherein the second region is a local region corresponding to the previous frame of the second close-up image on the previous frame of the second panoramic image.

[0032] In one possible implementation, determining the second close-up image based on the second magnified image and the second difference image includes: determining the difference / sum of color values ​​of each pixel based on the color values ​​of each pixel in the second difference image and the color values ​​of each pixel in the second magnified image; and obtaining the second close-up image based on the difference / sum of color values ​​of each pixel.

[0033] Fourthly, embodiments of this application provide an electronic device, including: a memory, a processor, a first camera, and a second camera; the processor is used to couple with the memory, read and execute instructions in the memory to implement the method described in the second aspect, the first camera is used to acquire a first panoramic image stream, the second camera is used to acquire a first close-up image stream, and the images of each close-up image in the first close-up image stream are close-ups of local areas on the synchronously acquired panoramic images.

[0034] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor; the processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method described in the third aspect.

[0035] Sixthly, embodiments of this application provide a readable storage medium storing a computer program; when executed, the computer program implements the methods described in the second or third aspect above.

[0036] The encoding and decoding method provided in this application embodiment is such that the close-up image is a close-up of a local area on the panoramic image. That is, the content of the close-up image and the content of the local area are the same, with only differences in image details. Therefore, the data volume of the difference image is very small, and the image jitter problem caused by concurrent I-frames will not occur. Attached Figure Description

[0037] Figure 1A This is a schematic diagram of an image captured by a panoramic camera in a shooting scene provided in an embodiment of this application;

[0038] Figure 1B Provided for the embodiments of this application Figure 1A A close-up image captured by a camera in the shooting scene shown;

[0039] Figure 1C A schematic diagram of a camera provided in an embodiment of this application;

[0040] Figure 2 System architecture diagram provided for embodiments of this application;

[0041] Figure 3 Application scenario diagrams provided for embodiments of this application;

[0042] Figure 4 A schematic diagram of the compressed flow provided in this application embodiment;

[0043] Figure 5 A software framework diagram 1 is provided for an embodiment of this application;

[0044] Figure 6 A schematic diagram of the compressed stream provided in the embodiments of this application. Figure 2 ;

[0045] Figure 7 An interaction diagram between the remote device 10 and the local device 20 provided in an embodiment of this application;

[0046] Figure 8 A flowchart for obtaining the first difference image provided in an embodiment of this application;

[0047] Figure 9 A flowchart for determining a first local region provided in an embodiment of this application;

[0048] Figure 10A Schematic diagram 1 for determining the first local region provided in the embodiments of this application;

[0049] Figure 10B The principle for determining the first local region provided in the embodiments of this application Figure 2 ;

[0050] Figure 11 A schematic diagram illustrating the principle of acquiring a first magnified image provided in an embodiment of this application;

[0051] Figure 12 A schematic diagram illustrating the principle of obtaining the first difference image as provided in an embodiment of this application;

[0052] Figure 13 A flowchart for obtaining a second close-up image provided in an embodiment of this application;

[0053] Figure 14A Schematic diagram 1 for determining the second local region provided in the embodiments of this application;

[0054] Figure 14B The principle for determining the second local region provided in the embodiments of this application Figure 2 ;

[0055] Figure 15 A schematic diagram illustrating the principle of acquiring a second magnified image as provided in an embodiment of this application;

[0056] Figure 16 A schematic diagram illustrating the principle of acquiring a second close-up image as provided in an embodiment of this application;

[0057] Figure 17 A schematic diagram of the encoding / decoding method provided in the embodiments of this application;

[0058] Figure 18 Software framework provided for embodiments of this application Figure 2 ;

[0059] Figure 19 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] First, some terms used in the embodiments of this application will be explained:

[0062] Panoramic cameras: Compared to close-up cameras, panoramic cameras have a wider field of view, capturing a broader image area. For example, Figure 1A The image shown is captured by a panoramic camera in a shooting scene.

[0063] Close-up camera: Compared to a panoramic camera, a close-up camera has a smaller field of view, resulting in a smaller area of ​​the captured image. For example, Figure 1B The image shown is a close-up shot captured by a camera in the same shooting scene.

[0064] For ease of explanation, in this embodiment, the image captured by the panoramic camera is referred to as a panoramic image, and the image captured by the close-up camera is referred to as a close-up image. The close-up image is a close-up of a local area on the panoramic image.

[0065] For example, see Figure 1C As shown, the field of view of the wide-angle camera 101 on the mobile phone is larger than that of the main camera 102, and the field of view of the main camera 102 is larger than that of the telephoto camera 103. Therefore, the wide-angle camera 101 can be used as a panoramic camera, and the main camera 102 and / or the telephoto camera 103 can be used as a close-up camera; or, the main camera 102 can be used as a panoramic camera, and the telephoto camera 103 can be used as a close-up camera. It should be noted that: Figure 1C The arrangement of the cameras in this embodiment is merely an example, and the embodiments described herein are not limited thereto.

[0066] I-frame: An image frame encoded using a spatial compression algorithm. When decoding an I-frame, only the data from the I-frame is needed to reconstruct the complete image.

[0067] P-frame: An image frame encoded using a temporal compression algorithm. Decoding a P-frame requires combining the image obtained from decoding the previous P-frame with the data of the current P-frame to reconstruct a complete image.

[0068] Figure 2 This is a system architecture diagram provided for an embodiment of this application. Figure 2 The system architecture shown includes a remote device 10 and a local device 20, which are connected via a network. The remote device 10 includes a panoramic camera and a close-up camera.

[0069] The following describes several possible forms of the remote device 10:

[0070] In one possible implementation, the remote device 10 is a separate terminal device equipped with a panoramic camera and a close-up camera. The terminal device includes, but is not limited to, a mobile phone. As described above, the wide-angle camera 101 on the terminal device can serve as a panoramic camera, and the telephoto camera 103 and / or the main camera 102 on the terminal device can serve as a close-up camera; or, the main camera 102 on the terminal device can serve as a panoramic camera, and the telephoto camera 103 on the terminal device can serve as a close-up camera. The terminal device is used to interact with the local device 20.

[0071] In another possible implementation, the remote device 10 includes a terminal device and at least one DSLR camera, which are connected. The camera on the terminal device can serve as a panoramic camera, and the camera on the at least one DSLR camera can serve as a close-up camera. The at least one DSLR camera sends the acquired close-up image stream to the terminal device, which is used to interact with the local device 20.

[0072] Alternatively, the camera on the terminal device can be used as a close-up camera, the camera on one of the at least one SLR cameras can be used as a panoramic camera, and the cameras on the other SLR cameras can be used as close-up cameras. At least one SLR camera sends the acquired panoramic image stream and close-up image stream to the terminal device, which is used to interact with the local device 20.

[0073] In another possible implementation, the remote device 10 includes multiple DSLR cameras interconnected. One of the DSLR cameras can function as a panoramic camera, while the cameras on the other DSLR cameras can function as close-up cameras. After the multiple DSLR cameras are connected, they can negotiate to determine which DSLR camera will interact with the local device 20. The other DSLR cameras send the captured image streams to that DSLR camera, which then interacts with the local device 20.

[0074] For example, remote device 10 includes two SLR cameras, namely a first SLR camera and a second SLR camera. The camera on the first SLR camera can be used as a panoramic camera, and the camera on the second SLR camera can be used as a close-up camera. After negotiation, it is determined that the first SLR camera is used to interact with the local device 20, and the second SLR camera can send the acquired close-up image stream to the first SLR camera, so that the first SLR camera and the local device 20 can interact.

[0075] Local device 20 is a device with display function, and the form of local device 20 includes, but is not limited to, mobile phones, tablets, laptops or televisions.

[0076] exist Figure 2 Under the system architecture shown, Figure 3 This is an application scenario diagram provided in an embodiment of this application. Figure 3 Taking a local device 20 as a laptop and a remote device 10 as a mobile phone as an example, when the local device 20 has no camera installed or has a limited number of cameras installed, the local device 20 can control the remote device 10 to activate the wide-angle camera 101, the main camera 102, and the telephoto camera 103 via the network, so that each camera starts capturing images. The remote device 10 can then send the image streams captured by each camera to the local device 20. Figure 3 The image stream captured by the wide-angle camera 101 is illustrated as a panoramic image stream 201, the image stream captured by the main camera 102 is illustrated as a close-up image stream 202, and the image stream captured by the telephoto camera 103 is illustrated as a close-up image stream 203. This allows the local device 20 to display multiple image streams. For example, the local device 20 can display the panoramic image stream 201 in area 301 of the screen, the close-up image stream 202 in area 302, and the close-up image stream 203 in area 303. This achieves the goal of the local device 20 obtaining multiple image streams using the multiple cameras of the remote device 10.

[0077] Figure 3 The remote device 10 sends a large amount of data to each image stream to the local device 20, but network bandwidth is limited. Therefore, the remote device 10 needs to compress each image stream. See [link / reference] Figure 4 As shown, since the wide-angle camera 101, the main camera 102, and the telephoto camera 103 acquire images simultaneously, when the remote device 10 compresses each image stream, the I-frames of the compressed stream often occur simultaneously. The I-frame has a large data volume, which will cause the data volume transmitted over the network to increase instantaneously, causing the display screen of the local device to jitter and affecting the user experience.

[0078] In some embodiments, the above-mentioned image shakiness problem can be solved in the following ways:

[0079] See Figure 5 As shown, a data sending module 11 and a data processing module 12 are configured in the remote device 10. The data sending module 11 can interact with the data processing module 12, and the data processing module 12 is also connected to each camera of the remote device 10. The data sending module 11 is used to monitor the current network bandwidth and send the current network bandwidth to the data processing module 12. After each camera on the remote device 10 acquires an image stream, it sends the image stream to the data processing module 12. Figure 2 The image streams are represented by streams 11, 21, and 31. When the data processing module 12 determines that the current network bandwidth is less than a preset value, it reduces the bit rate of each compressed image stream and then transmits the compressed streams of each image stream to the sending module 11, which then sends them to the local device 20.

[0080] A data recovery module 21, a frame rate stabilization module 22, and a data consumption module 23 are configured in the local device 20, and these modules are connected sequentially. After receiving the compressed streams from each image stream, the data recovery module 21 decodes the compressed streams to obtain... Figure 2 Streams 12, 22, and 32. The data recovery module 21 transmits the decoded image streams to the stable frame rate module 22. The stable frame rate module 22 ensures that the number of frames in each image stream sent to the data consumption module 23 within the same time period is the same. The data consumption module 23 can be a display module, which is used to display the image streams sent by the stable frame rate module 22.

[0081] In the above embodiments, because the data processing module 12 reduces the bit rate of each image stream, the image quality of streams 11, 21 and 31 obtained by the data recovery module 21 after decoding the compressed streams of each image stream is reduced, which affects the user experience.

[0082] This application provides an encoding method, combined with... Figure 3 In the scenario shown, when the remote device 10 encodes the panoramic image stream 201, the close-up image stream 202, and the close-up image stream 203, see [reference needed]. Figure 6 As shown, for the panoramic image stream 201, a compressed stream of the panoramic image stream 201 is obtained by encoding it using a traditional method. For the close-up image streams 202 and 203, instead of directly encoding them, the correlation between the close-up images and the panoramic images is utilized to calculate the difference image between magnified images of local regions in the close-up and panoramic images. This difference image stream is then encoded to obtain a compressed stream of the difference image stream. Figure 6In this embodiment, difference image stream 401 represents the difference image stream corresponding to close-up image stream 202, and difference image stream 402 represents the difference image stream corresponding to close-up image stream 203. Since the close-up image is a close-up of a local area on the panoramic image, meaning the content of the close-up image and the content of that local area are the same, differing only in details, the data size of the difference image is very small. The P' frame obtained by encoding the difference image is smaller than the P frame obtained by traditional encoding, thus avoiding the image jitter problem caused by concurrent I-frames. Furthermore, this embodiment does not reduce the bitrate of each image stream, and... Figure 5 Compared to the illustrated embodiment, the image quality is improved.

[0083] based on Figure 2 The system architecture shown below details the interaction process between remote device 10 and local device 20.

[0084] Figure 7 An interaction diagram of a remote device 10 and a local device 20 provided in an embodiment of this application.

[0085] Specifically, when the remote device 10 is a standalone terminal device, it interacts with the local device 20. When the remote device 10 includes a terminal device and at least one SLR camera, it interacts with the local device 20. When the remote device 10 includes multiple SLR cameras, the SLR camera among the multiple SLR cameras used to interact with the local device 20 interacts with the local device 20. Figure 3 The interactive diagram shown includes the following steps:

[0086] S701, Local device 20 sends shooting instructions to remote device 10.

[0087] After receiving the shooting command, the remote device 10 (S702) activates the panoramic camera and the close-up camera.

[0088] In one possible implementation, a camera application is installed on the local device 20, and a corresponding remote camera agent is installed on the remote device 10. When a user needs to obtain multiple image streams using multiple cameras on the remote device 10, they can trigger a shooting command on the camera application. The camera application sends the shooting command to the remote camera agent. After receiving the shooting command, the remote camera agent turns on the panoramic camera and the close-up camera, and the panoramic camera and the close-up camera begin to synchronously acquire images.

[0089] In another possible implementation, the local device 20 and the remote device 10 have the same video calling application installed. After the video call between the local device 20 and the remote device 10 is connected, the user can trigger a shooting command on the video calling application installed on the local device 20. The video calling application installed on the local device 20 sends the shooting command to the video calling application installed on the remote device 10. After receiving the shooting command, the video calling application installed on the remote device 10 turns on the panoramic camera and the close-up camera, and the panoramic camera and the close-up camera start to capture images synchronously.

[0090] For ease of distinction, the image stream captured by the panoramic camera will be referred to as the first panoramic image stream, and the image stream captured by the close-up camera will be referred to as the first close-up image stream.

[0091] Combination Figure 3 In the scenario shown, after receiving the shooting command, the remote device 10 activates the wide-angle camera 101, the main camera 102, and the telephoto camera 103.

[0092] S703, remote device 10 encodes the first panoramic image to obtain the first compressed image.

[0093] The aforementioned first panoramic image is any image in the first panoramic image stream. If it is necessary to encode the first panoramic image as an I-frame, a spatial compression algorithm can be used. If it is necessary to encode the first panoramic image as a P-frame, a temporal compression algorithm can be used.

[0094] exist Figure 3 In the scenario shown, the first panoramic image stream is panoramic image stream 201.

[0095] S704, the remote device 10 obtains a first difference image based on the first panoramic image and the first close-up image.

[0096] By execution Figure 8 The method shown is used to obtain the first difference image. For ease of explanation, the image that is synchronously acquired with the first panoramic image in the first close-up image stream is called the first close-up image. The first close-up image is a close-up of a local area on the first panoramic image.

[0097] Figure 8 The method shown specifically includes:

[0098] S10. Based on the relative positions of the panoramic camera and the close-up camera, the zoom level of the panoramic camera, and the zoom level of the close-up camera, determine the first local region corresponding to the first close-up image on the first panoramic image.

[0099] See Figure 9 As shown, the first local region can be determined in the following way:

[0100] S20. Determine whether the relative positions of the panoramic camera and the close-up camera have changed.

[0101] Specifically, the first relative position is obtained, which is the relative position of the panoramic camera and the close-up camera when they capture the previous frame of the first panoramic image and the previous frame of the first close-up image, respectively. The first relative position is compared with the current relative position, and the comparison result is used to determine whether the relative position of the panoramic camera and the close-up camera has changed.

[0102] If the relative positions of the panoramic camera and the close-up camera change, execute S21; otherwise, execute S22.

[0103] S21. Use an image recognition algorithm to determine the first local region.

[0104] S22. Determine whether the zoom ratio of the panoramic camera and the close-up camera has changed.

[0105] Specifically, the zoom level of the panoramic camera in the previous frame of the first panoramic image is obtained, and the current zoom level of the panoramic camera is compared with that zoom level. Based on the comparison result, it is determined whether the zoom level of the panoramic camera has changed. The zoom level of the close-up camera in the previous frame of the first close-up image is obtained, and the current zoom level of the close-up camera is compared with that zoom level. Based on the comparison result, it is determined whether the zoom level of the close-up camera has changed.

[0106] If the zoom level of either the panoramic camera or the close-up camera changes, then execute S23; if the zoom level of neither the panoramic camera nor the close-up camera changes, then execute S24.

[0107] S23. Determine the first local area based on the current zoom level of the panoramic camera and the current zoom level of the close-up camera.

[0108] In the scenario where the remote device 10 is a separate terminal device, the first panoramic image and the first close-up image are images with the center point aligned. The first local area can be determined based on the current zoom level of the panoramic camera, the current zoom level of the close-up camera, the size of the first close-up image, and the center point of the first panoramic image.

[0109] The following example illustrates this:

[0110] See Figure 10AAs shown, assuming the current zoom level of the panoramic camera is A times, the current zoom level of the close-up camera is B times, the length of the first close-up image is L, and the width of the first close-up image is W, then the first local region 501 is: a region with the center point of the first panoramic image, a length of L*(A / B), and a width of W*(A / B).

[0111] For scenarios where the remote device 10 includes a terminal device and at least one SLR camera, or for scenarios where the remote device 10 includes multiple SLR cameras, a first local region can be determined based on the current zoom level of the panoramic camera, the current zoom level of the close-up camera, and the first region. The first region is the local region corresponding to the previous frame of the first close-up image on the previous frame of the first panoramic image.

[0112] The following example illustrates this:

[0113] See Figure 10B As shown, assuming the first region 502 is Figure 10B For a region with a length of L and a width of W, if the current zoom level of the panoramic camera is A and the current zoom level of the close-up camera is B, then the first local region 501 is: the region corresponding to the length of the first region 502 magnified by A / B times, and the region corresponding to the width of the first region 502 magnified by A / B times.

[0114] S24. Based on the first region, determine the first local region, where the first region is the local region corresponding to the previous frame of the first close-up image on the previous frame of the first panoramic image.

[0115] For example: See Figure 10B As shown, assuming the first region 502 is a region with length L and width W in the previous frame of the first panoramic image, find the region in the first panoramic image that corresponds to the position of the first region 502, and take this region as the first local region mentioned above.

[0116] After obtaining the first local region, perform the following steps:

[0117] S11. Extract the first local area determined in S10 from the first panoramic image to obtain the first local image.

[0118] S12. Adjust the size of the first partial image to be the same as the size of the first close-up image to obtain the first magnified image.

[0119] The following example illustrates this:

[0120] See Figure 11As shown, through S10, the first local region 501 can be determined as: a region with a center point of the center point of the first panoramic image, a length of L*(A / B), and a width of W*(A / B). This region 501 can be extracted from the first panoramic image to obtain the first local image. The length of the first local image is L*(B / A), and the width is W*(B / A). By enlarging the length of the first local image to L and the width of the first local image to W, the first enlarged image is obtained.

[0121] S13. Determine the first difference image based on the first magnified image and the first close-up image.

[0122] In one possible implementation, the difference in color values ​​of each pixel can be determined based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image; and a first difference image can be obtained based on the difference in color values ​​of each pixel.

[0123] For example, on the one hand, the YUV values ​​of each pixel in the first magnified image are extracted, and on the other hand, the YUV values ​​of each pixel in the first close-up image are extracted. Then, the YUV values ​​of each pixel in the first magnified image are subtracted from the YUV values ​​of the corresponding pixels in the first close-up image to obtain the YUV value of each pixel. Based on the YUV value of each pixel, the first difference image is rendered.

[0124] The following example illustrates this:

[0125] See Figure 12 As shown, after obtaining the first magnified image via S3034, the YUV values ​​of each pixel in the first magnified image are extracted. Assume the YUV values ​​of the pixel in the first row and first column are (A1, B1, C1). On the other hand, the YUV values ​​of each pixel in the first close-up image are extracted. Assume the YUV values ​​of the pixel in the first row and first column are (A2, B2, C2). The YUV values ​​of each pixel in the first magnified image are subtracted from the YUV values ​​of the corresponding pixels in the first close-up image to obtain the YUV value of each pixel. For example, the YUV values ​​of the pixel in the first row and first column are (A0 = A1 - A2, B0 = B1 - B2, C0 = C1 - C2). After obtaining the YUV value of each pixel, a first difference image is rendered based on the YUV value of each pixel.

[0126] It should be noted that the YUV values ​​of each pixel in the first close-up image corresponding to the first panoramic image can also be subtracted from the YUV values ​​of the corresponding pixels in the first magnified image. The above example is only one example and does not constitute a limitation on this application.

[0127] exist Figure 3In the scenario shown, the first close-up image stream includes close-up image stream 202 and close-up image stream 203. Based on the panoramic image stream 201 and the close-up image stream 202, the difference image stream 401 corresponding to the close-up image stream 202 can be obtained. Similarly, based on the panoramic image stream 201 and the close-up image stream 203, the difference image stream 402 corresponding to the close-up image stream 203 can be obtained. The specific process is described in S10-S13, and will not be repeated here.

[0128] S705. Encode the first difference image to obtain the second compressed image.

[0129] In one possible implementation, a spatial compression algorithm can be used to encode the first difference image.

[0130] S706, the remote device 10 sends the first compressed image and the second compressed image to the local device 20.

[0131] S707 and local device 20 decode the first compressed image to obtain the second panoramic image.

[0132] In one possible implementation, if the first compressed image is an I-frame, the complete image is reconstructed based on the data of the I-frame; if the first compressed image is a P-frame, the complete image is reconstructed based on the image obtained after decoding the previous P-frame and the data of the current P-frame.

[0133] S708 and local device 20 decode the second compressed image to obtain the second difference image.

[0134] In one possible implementation, a complete image is reconstructed based on the data of an image frame encoded using a spatial compression algorithm.

[0135] S709, Local device 20 acquires a second close-up image based on the second panoramic image and the second difference image.

[0136] For each frame of the second difference image stream, it can be achieved by performing... Figure 13 The method shown is used to obtain the corresponding second close-up image. For ease of explanation, the image in the second difference image stream is referred to as the second difference image, and the panoramic image corresponding to the image in the second panoramic image stream is referred to as the second panoramic image.

[0137] Figure 13 The method shown specifically includes:

[0138] S30. Determine the second local region corresponding to the second close-up image on the second panoramic image.

[0139] When the detection result of S20 indicates that the relative positions of the panoramic camera and the close-up camera have changed, the remote device 10 can send the first compressed image and the second compressed image to the local device 20 in S706, and at the same time, send the information of the first local area determined in S21 to the local device 20. The local device 20 determines the second local area based on the information of the first local area.

[0140] When the detection result in S20 indicates that the relative positions of the panoramic camera and the close-up camera have not changed, and the detection result in S22 indicates that the zoom levels of the panoramic camera and the close-up camera have changed, the remote device 10, while sending the first compressed image and the second compressed image to the local device 20 in S706, may also send information about the first local area determined in S23 or camera information to the local device 20. The camera information includes the current zoom level of the panoramic camera and the current zoom level of the close-up camera. If the remote device 10 sends information about the first local area determined in S23 to the local device 20, the local device 20 determines the second local area based on this information.

[0141] For example: Suppose remote device 10 sends a message to local device 20. Figure 10A Based on the information of the first local region 501, the local device 20 can find the region corresponding to the position of the first local region 501 on the second panoramic image and use that region as the second local region.

[0142] If the remote device 10 sends camera information to the local device 20, the local device 20 determines the second local area in the following way:

[0143] In one possible implementation, a second local region can be determined based on the current zoom level of the panoramic camera, the current zoom level of the close-up camera, the size of the second difference image, and the center point of the second panoramic image, wherein the size of the second close-up image is equal to the size of the second difference image.

[0144] The following example illustrates this:

[0145] See Figure 14A As shown, assume the current zoom level of the panoramic camera is A times and the current zoom level of the close-up camera is B times. The length of the second difference image is L, and the width of the second difference image is W. Then the second local region 503 is: a region with a length of L*(A / B) and a width of W*(A / B), centered at the center point of the second panoramic image.

[0146] In another possible implementation, a second local region can be determined based on the current zoom level of the panoramic camera, the current zoom level of the close-up camera, and the second region. The second region is the local region corresponding to the previous frame of the second close-up image on the previous frame of the second panoramic image.

[0147] The following example illustrates this:

[0148] See Figure 14B As shown, assuming the second region 504 is Figure 14B For a region with a length of L and a width of W, if the current zoom level of the panoramic camera is A and the current zoom level of the close-up camera is B, then the second local region 503 is the region whose length is magnified by A / B times and whose width is magnified by A / B times for the second region 504.

[0149] After obtaining the second local region, perform the following steps:

[0150] S31. Extract the second local region from the second panoramic image to obtain the second local image.

[0151] S32. Adjust the size of the second local image to be the same as the size of the second difference image to obtain the second magnified image.

[0152] The following example illustrates this:

[0153] See Figure 15 As shown, the second local region 503 can be determined by S30 as: a region with a center point of the center point of the second panoramic image, a length of L*(A / B), and a width of W*(A / B). This region can be extracted from the second panoramic image to obtain the second local image. The length of the second local image is L*(B / A), and the width is W*(B / A). By enlarging the length of the second local image to L and the width of the second local image to W, the second enlarged image is obtained.

[0154] S33. Determine the second close-up image based on the second magnified image and the second difference image.

[0155] In one possible implementation, the difference / sum of color values ​​of each pixel can be determined based on the color values ​​of each pixel in the difference image and the color values ​​of each pixel in the second magnified image; and the second close-up image can be obtained based on the difference / sum of color values ​​of each pixel.

[0156] For example, on one hand, the YUV values ​​of each pixel in the second magnified image are extracted, and on the other hand, the YUV values ​​of each pixel in the second difference image are extracted. Then, the YUV values ​​of each pixel in the second magnified image are subtracted from the YUV values ​​of the corresponding pixels in the second difference image to obtain the YUV value of each pixel. Based on the YUV values ​​of the corresponding pixels in the two images, the second close-up image is rendered.

[0157] The following example illustrates this:

[0158] See Figure 16 As shown, after obtaining the second magnified image via S32, the YUV values ​​of each pixel in the second magnified image are extracted. Assume the YUV values ​​extracted to the pixel in the first row and first column are (A1, B1, C1). On the other hand, the YUV values ​​of each pixel in the second difference image are extracted. Assume the YUV values ​​extracted to the pixel in the first row and first column are (A0, B0, C0). The YUV values ​​of each pixel in the first magnified image are subtracted from the YUV values ​​of the corresponding pixels in the second difference image to obtain the YUV value of each pixel. For example, the YUV values ​​of the pixel in the first row and first column are (A2 = A1 - A0, B2 = B1 - B0, C2 = C1 - C0). After obtaining the YUV value of each pixel, the second close-up image is rendered based on the YUV value of each pixel.

[0159] exist Figure 3 In the scenario shown, after receiving the compressed streams of panoramic image stream 201, difference image stream 401, and difference image stream 402, the local device 20 decodes the compressed stream of panoramic image stream 201 to obtain a decoded panoramic image stream, which is designated as the first image stream. Decoding the compressed stream of difference image stream 401 yields a decoded difference image stream. Based on the first image stream and the decoded difference image stream, the second image stream is obtained via steps S30-S33. Decoding the compressed stream of difference image stream 402 yields a decoded difference image stream. Based on the first image stream and the decoded difference image stream, the third image stream is obtained via steps S30-S33. The local device can display these three image streams on a screen for user viewing, thus achieving the goal of obtaining multiple image streams using multiple cameras from the remote device 10.

[0160] It should be noted that when acquiring the third image stream, the third image stream can also be acquired based on the second image stream and the decoded difference image stream. This application embodiment is not limited to this.

[0161] The encoding / decoding method provided in this application embodiment, because the close-up image is a close-up of a local area on a panoramic image, that is, the content of the close-up image and the content of that local area are consistent, with only differences in image details, therefore the data volume of the difference image is very small, and the image jitter problem caused by I-frame concurrency will not occur. Furthermore, this application embodiment does not reduce the bitrate of each image stream, and... Figure 5 Compared to the illustrated embodiment, the image quality is improved.

[0162] Figure 17 This is a schematic diagram of another embodiment provided in this application. After the remote device 10 acquires a first panoramic image and a first close-up image, it obtains a first compressed image by executing S703. It obtains a first partial image by executing S10 and S11. Based on the first partial image, it obtains a first magnified image by executing S12. Based on the first magnified image and the first close-up image, it obtains a first difference image by executing S13. After the local device 20 receives the first compressed image and the second compressed image sent by the remote device 10, it obtains a second panoramic image by executing S707 and a second difference image by executing S708. It obtains a second partial image by executing S30 and S31. Based on the second partial image, it obtains a second magnified image by executing S32. Based on the second magnified image and the second difference image, it obtains a second close-up image by executing S33.

[0163] Figure 18 A software framework diagram provided for this application. See also... Figure 18 As shown, the remote device 10 includes, but is not limited to, a remote camera agent 13, a camera control module 14, cameras 1-3, a first cropping and magnification module 15, a difference calculation module 16, and an encoding module 17. The local device 20 includes, but is not limited to, a camera application 24, a camera control module 25, a decoding module 26, a second cropping and magnification module 27, a close-up image recovery module 28, a stable frame rate output module 29, and a data output module 30.

[0164] In one possible implementation, to obtain multiple image streams using multiple cameras on remote device 10, the user can trigger a shooting command on camera application 24. Camera application 24 sends the shooting command to camera control module 25, which then sends it to remote device agent 13. Remote device agent 13 further sends the shooting command to camera control module 14, which then activates cameras 1-3. Camera 1 begins capturing panoramic images, while cameras 2 and 3 begin capturing close-up images. Camera 1 can be a wide-angle camera, and cameras 2 and 3 can be a main camera and a telephoto camera, respectively.

[0165] In the remote device 10, the encoding module 17 encodes the first panoramic image to obtain a first compressed image. The first cropping and magnification module 15 determines a first local region corresponding to the first close-up image on the first panoramic image based on the relative positions of the panoramic camera and the close-up camera, the zoom level of the panoramic camera, and the zoom level of the close-up camera. It then crops the first local region from the first panoramic image to obtain a first local image and adjusts the size of the first local image to be the same as the size of the first close-up image to obtain a first magnified image. The difference calculation module 16 determines a first difference image based on the first magnified image and the first close-up image. The encoding module 17 further encodes the first difference image to obtain a second compressed image and sends the first compressed image and the second compressed image to the local device 20.

[0166] In local device 20, decoding module 26 decodes the first compressed image to obtain the second panoramic image, and decodes the second compressed image to obtain the second difference image. Second cropping and magnification module 27 determines the second local region corresponding to the second close-up image on the second panoramic image, crops the second local region from the second panoramic image to obtain the second local image, and adjusts the size of the second local image to be the same as the size of the second difference image to obtain the second magnified image. Close-up image restoration module 28 determines the second close-up image based on the second magnified image and the second difference image. Stable frame rate output module 29 controls the number of frames sent to each image stream to data output module 30 to be the same within the same time period. Data output module 30 sends each image stream to the camera application, and camera application 24 displays each image stream.

[0167] Figure 19 A schematic diagram of the electronic device 100 is shown. Both the remote device 10 and the local device 20 can be adopted. Figure 15 The structure shown in the diagram.

[0168] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, button 190, motor 191, indicator 192, camera 193, and display screen 194.

[0169] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0170] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0171] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0172] The processor 110 may also include a memory for storing instructions and data. These instructions include those corresponding to the methods provided in the embodiments of this application. When the processor 110 executes these instructions, it can implement the steps in each flowchart.

[0173] In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has just used or that are used repeatedly. If processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of processor 110, and thus improves system efficiency.

[0174] Remote device 10 adopts Figure 19 In the structure shown, the video encoder includes Figure 18 Chinese encoding module 17. Local device 20 uses... Figure 19 In the structure shown, the video decoder includes Figure 18 Decoding module 26.

[0175] Camera 193 may include a wide-angle camera, a main camera, and / or a telephoto camera.

[0176] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A communication system, characterized in that, The communication system includes: a first device and a second device. The first device includes: a first camera and a second camera. The first camera is used to acquire a first panoramic image stream, and the second camera is used to acquire a first close-up image stream. The images of each close-up image in the first close-up image stream are close-ups of local areas on the panoramic images acquired synchronously. The first device is used for: Upon receiving the shooting command sent by the second device, the first camera and the second camera are turned on; Encode the first panoramic image to obtain a first compressed image, wherein the first panoramic image is any image in the first panoramic image stream; Determining the first local region corresponding to the first close-up image on the first panoramic image includes: determining whether the relative position of the first camera and the second camera has changed; if the relative position has changed, then using an image recognition algorithm to determine the first local region; if the relative position has not changed, then determining whether the zoom factor of the first camera and the zoom factor of the second camera have changed; if the zoom factor of the first camera and / or the zoom factor of the second camera has changed, then determining the first local region based on the current zoom factor of the first camera and the current zoom factor of the second camera; if neither the zoom factor of the first camera nor the zoom factor of the second camera has changed, then determining the first local region based on a first region, wherein the first region is the local region corresponding to the previous frame image of the first close-up image on the previous frame image of the first panoramic image; The first local area is cropped from the first panoramic image to obtain the first local image; The size of the first partial image is adjusted to be the same as the size of the first close-up image to obtain the first magnified image; Based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image, the difference in color values ​​of each pixel is determined; based on the difference in color values ​​of each pixel, a first difference image is obtained, wherein the first close-up image is an image that was synchronously acquired with the first panoramic image in the first close-up image stream; The first difference image is encoded to obtain the second compressed image; Send the first compressed image and the second compressed image to the second device; The second device is used for: Decode the first compressed image to obtain the second panoramic image; Decode the second compressed image to obtain the second difference image; A second close-up image is obtained based on the second panoramic image and the second difference image.

2. The communication system according to claim 1, characterized in that, The first device is specifically used for: The first local region is determined based on the center point of the first panoramic image, the size of the first close-up image, the current zoom level of the first camera, and the current zoom level of the second camera.

3. The communication system according to claim 1, characterized in that, The first device is specifically used for: The first local area is determined based on the first region, the current zoom level of the first camera, and the current zoom level of the second camera.

4. The communication system according to any one of claims 1-3, characterized in that, The second device is specifically used for: Determine the second local region corresponding to the second close-up image on the second panoramic image; The second local region is cropped from the second panoramic image to obtain the second local image; The size of the second local image is adjusted to be the same as the size of the second difference image to obtain the second magnified image; The second close-up image is determined based on the second magnified image and the second difference image.

5. The communication system according to claim 4, characterized in that, The second device is also used for: Receive information about the first local area sent by the first device; The second device is specifically used for: The second local region is determined based on the information of the first local region.

6. The communication system according to claim 4, characterized in that, The second device is also used for: Receive the current zoom level of the first camera and the current zoom level of the second camera sent by the first device; The second device is specifically used for: The second local region is determined based on the current zoom level of the first camera and the current zoom level of the second camera.

7. The communication system according to claim 6, characterized in that, The second device is specifically used for: The second local region is determined based on the center point of the second panoramic image, the size of the second difference image, the current zoom level of the first camera, and the current zoom level of the second camera.

8. The communication system according to claim 6, characterized in that, The second device is specifically used for: Based on the second region, the current zoom level of the first camera, and the current zoom level of the second camera, the second local region is determined. The second region is the local region corresponding to the previous frame of the second close-up image on the previous frame of the second panoramic image.

9. The communication system according to any one of claims 5-8, characterized in that, The first device is specifically used for: Based on the color values ​​of each pixel in the second difference image and the color values ​​of each pixel in the second magnified image, determine the difference / sum value of the color values ​​of each pixel; The second close-up image is obtained based on the difference / sum of the color values ​​of each pixel.

10. An encoding method, characterized in that, The method is applied to a first device, the first device comprising: a first camera and a second camera, the first camera being used to capture a first panoramic image stream, and the second camera being used to capture a first close-up image stream, wherein each close-up image in the first close-up image stream is a close-up of a local area on a synchronously captured panoramic image; the method includes: Upon receiving a shooting command from the second device, the first camera and the second camera are activated; Encode the first panoramic image to obtain a first compressed image, wherein the first panoramic image is any image in the first panoramic image stream; Determining the first local region corresponding to the first close-up image on the first panoramic image includes: determining whether the relative position of the first camera and the second camera has changed; if the relative position has changed, then using an image recognition algorithm to determine the first local region; if the relative position has not changed, then determining whether the zoom factor of the first camera and the zoom factor of the second camera have changed; if the zoom factor of the first camera and / or the zoom factor of the second camera has changed, then determining the first local region based on the current zoom factor of the first camera and the current zoom factor of the second camera; if neither the zoom factor of the first camera nor the zoom factor of the second camera has changed, then determining the first local region based on a first region, wherein the first region is the local region corresponding to the previous frame image of the first close-up image on the previous frame image of the first panoramic image; The first local area is cropped from the first panoramic image to obtain the first local image; The size of the first partial image is adjusted to be the same as the size of the first close-up image to obtain the first magnified image; Based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image, the difference in color values ​​of each pixel is determined; based on the difference in color values ​​of each pixel, a first difference image is obtained, wherein the first close-up image is an image that was synchronously acquired with the first panoramic image in the first close-up image stream; The first difference image is encoded to obtain the second compressed image; The first compressed image and the second compressed image are sent to the second device.

11. The method according to claim 10, characterized in that, Determining the first local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: The first local region is determined based on the center point of the first panoramic image, the size of the first close-up image, the current zoom level of the first camera, and the current zoom level of the second camera.

12. The method according to claim 10, characterized in that, Determining the first local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: The first local area is determined based on the first region, the current zoom level of the first camera, and the current zoom level of the second camera.

13. A decoding method, characterized in that, include: Decode the first compressed image to obtain the second panoramic image; Decode the second compressed image to obtain the second difference image; Determine the second local region corresponding to the second close-up image on the second panoramic image; The second local region is cropped from the second panoramic image to obtain the second local image; The size of the second local image is adjusted to be the same as the size of the second difference image to obtain the second magnified image; The second close-up image is determined based on the second magnified image and the second difference image; The first compressed image is obtained by encoding the first panoramic image, and the first panoramic image is any image in the first panoramic image stream; The second compressed image is obtained by encoding the first difference image, which is determined in the following way: A first local region corresponding to a first close-up image on the first panoramic image is determined, wherein the first close-up image is an image synchronously acquired in the first close-up image stream along with the first panoramic image. The determination method includes: determining whether the relative position of the first camera and the second camera has changed; if the relative position has changed, then using an image recognition algorithm to determine the first local region; if the relative position has not changed, then determining whether the zoom factor of the first camera and the zoom factor of the second camera have changed; if the zoom factor of the first camera and / or the zoom factor of the second camera has changed, then determining the first local region based on the current zoom factor of the first camera and the current zoom factor of the second camera; if the zoom factor of the first camera and the zoom factor of the second camera have not changed, then determining the first local region based on a first region, wherein the first region is a local region corresponding to the previous frame image of the first close-up image on the previous frame image of the first panoramic image, wherein the first camera is used to acquire the first panoramic image stream, and the second camera is used to acquire the first close-up image stream. The first local area is cropped from the first panoramic image to obtain the first local image; The size of the first partial image is adjusted to be the same as the size of the first close-up image to obtain the first magnified image; Based on the color values ​​of each pixel in the first close-up image and the color values ​​of each pixel in the first magnified image, the difference in color values ​​of each pixel is determined, and the first difference image is obtained based on the difference.

14. The method according to claim 13, characterized in that, The method further includes: Receive information about a first local area sent by the first device; Determining the second local region corresponding to the second close-up image on the second panoramic image includes: The second local region is determined based on the information from the first local region.

15. The method according to claim 13, characterized in that, The method further includes: Receive the current zoom level of the first camera and the current zoom level of the second camera sent by the first device; Determining the second local region corresponding to the second close-up image on the second panoramic image includes: The second local region is determined based on the current zoom level of the first camera and the current zoom level of the second camera.

16. The method according to claim 15, characterized in that, Determining the second local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: The second local region is determined based on the center point of the second panoramic image, the size of the second difference image, the current zoom level of the first camera, and the current zoom level of the second camera.

17. The method according to claim 15, characterized in that, Determining the second local region based on the current zoom level of the first camera and the current zoom level of the second camera includes: Based on the second region, the current zoom level of the first camera, and the current zoom level of the second camera, the second local region is determined. The second region is the local region corresponding to the previous frame of the second close-up image on the previous frame of the second panoramic image.

18. The method according to any one of claims 13-17, characterized in that, Determining the second close-up image based on the second magnified image and the second difference image includes: Based on the color values ​​of each pixel in the second difference image and the color values ​​of each pixel in the second magnified image, determine the difference / sum value of the color values ​​of each pixel; The second close-up image is obtained based on the difference / sum of the color values ​​of each pixel.

19. An electronic device, characterized in that, include: Memory, processor, first camera, and second camera; The processor is used to couple with the memory, read and execute instructions in the memory to implement the method of any one of claims 10-12, wherein the first camera is used to acquire a first panoramic image stream, the second camera is used to acquire a first close-up image stream, and the images of each close-up image in the first close-up image stream are close-ups of local areas on the synchronously acquired panoramic images.

20. An electronic device, characterized in that, include: Memory and processor; The processor is configured to couple with the memory, read and execute instructions in the memory to implement the method of any one of claims 13-18.

21. A readable storage medium, characterized in that, The readable storage medium stores a computer program; when the computer program is executed, it implements the method described in any one of claims 10-12, or the method described in any one of claims 13-18.

Images