Shooting method, terminal equipment and storage medium

Abstract

The invention discloses a shooting method, terminal equipment and a storage medium, and belongs to the technical field of images. The method is applied to the terminal equipment and comprises the steps that a first interface is displayed, and the first interface comprises a shooting preview picture; in the non-zero second delay scene, the first image generated in response to the shooting operation of the user is inconsistent with the preview image in the shooting preview picture. Obtaining jitter offset information of a camera in the terminal device from the moment when the shooting operation is received to the frame outlet moment of the first image (i.e., shutter delay time), the jitter offset information reflecting the jitter degree of the camera in the shutter delay time; wherein the first image is an image generated by the camera in a non-zero second delay mode. The first image is corrected according to the jitter offset information, the target image is obtained, the target image is the shooting result, the target image output after final correction corresponds to the preview image, the view field jump problem is reduced, and the shooting experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a shooting method, a terminal device, and a storage medium. Background Art

[0002] With the development of electronic technology, terminal devices (such as mobile phones, tablets, or smart wearable devices) have become increasingly functional. For example, most terminal devices have a camera function. When a user uses a camera application on a handheld terminal device to take a photo, the preview window of the camera application typically provides the user with multiple consecutive preview frames, allowing the user to directly observe changes in the preview image. After the user clicks the photo control in the photo application, the captured target image is displayed to the user.

[0003] In most scenarios, camera apps use a zero-shutter-lag (ZSL) framing strategy (shoot-and-see), meaning the preview image displayed when the camera is clicked is the same frame as the captured target image. However, in some scenarios, a non-ZSL framing strategy is required, where the target image is output after a certain period of time after the camera is clicked. For example, to achieve a better signal-to-noise ratio or due to differences in the processing flow between the preview and camera streams, the interval between the final frame (i.e., the target image) and the frame displayed when the user clicks the camera (i.e., the preview image) is typically greater than 200 milliseconds (ms).

[0004] When users use handheld terminal devices to take photos, jitter is inevitable. Using a non-ZSL frame output strategy will cause the preview image captured by the camera when clicking the photo control to shift after the jitter, becoming the camera's frame output image. This causes the preview image to be inconsistent with the final target image, resulting in field of view (FOV) jump problems, affecting the user's photo-taking experience. Summary of the Invention

[0005] This application provides a shooting method, terminal device, and storage medium, so that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's shooting experience. The technical solution is as follows:

[0006] In a first aspect, a shooting method is provided, which is applied to a terminal device, and the method includes: displaying a first interface, the first interface including a shooting preview screen; in a non-zero-second delay scenario, the first image generated in response to the user's shooting operation is inconsistent with the preview image in the shooting preview screen. The jitter offset information of the camera in the terminal device between the moment the shooting operation is received and the moment the first image is framed (i.e., the shutter delay time) is obtained, and the jitter offset information reflects the degree of jitter of the camera within the shutter delay time; wherein the first image is an image generated by the camera in the non-zero-second delay mode. The first image is corrected according to the jitter offset information to obtain a target image, which is the shooting result. By correcting the jitter within the shutter delay time, the target image output after the final correction corresponds to the preview image, thereby reducing the field of view jump problem and improving the user's shooting experience.

[0007] On the one hand, when the camera is shooting at its base magnification, the position of the camera can be offset according to the jitter offset information and then reframed, and the reframed image corresponds to the preview image in the shooting preview screen, and the reframed image is used as the target image to achieve correction of the first image. On the other hand, when shooting at a high magnification (i.e., zoomed in) and the jitter is not severe, the cropping frame on the first image can be offset according to the jitter offset information and then cropped on the first image and output the target image to achieve correction of the first image. On the other hand, when shooting at a high magnification (i.e., zoomed in) and the jitter is severe, the position of the camera can be offset according to the jitter offset information and then reframed, and the cropping frame of the reframed image can be offset according to the jitter offset information and then cropped on the reframed image and output the target image to achieve correction of the first image, so that the final corrected target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0008] In one possible implementation, the above-mentioned step of correcting the first image can also be implemented in the following manner. The position of the camera is controlled to be adjusted to a position opposite to the jitter offset information, so that the position of the adjusted camera corresponds to the position of the camera when the shooting operation is received. The target image is determined based on the second image generated by the adjusted camera to achieve correction of the first image. In the photo stream, the first image and the second image are adjacent, and the second image is the next frame of the first image. That is to say, by adjusting the position of the camera and re-framing, the FOV jump problem can be reduced and the user's photo experience can be improved.

[0009] In a possible implementation, when the shooting magnification of the camera is the basic magnification, it is necessary to re-frame and control the position of the camera to be adjusted to a position opposite to the jitter offset information.

[0010] If the camera's shooting magnification is high (i.e., greater than the base magnification) and the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame, this indicates significant jitter and requires re-framing. The camera's position is adjusted to a position opposite to the jitter offset information. The target cropping frame is obtained by transforming the first cropping frame based on the jitter offset information. The center point of the first cropping frame corresponds to the center point of the first image, and the size of the first cropping frame corresponds to the shooting magnification.

[0011] In the embodiment of the present application, at the basic magnification, high magnification and large degree of jitter, it is necessary to re-frame and control the position of the camera to adjust to the position opposite to the jitter offset information, and then determine the target image based on the second image generated by the adjusted camera, so that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0012] In one possible implementation, when the camera's shooting magnification is the basic magnification, there is no need to crop, and the second image generated by the adjusted shooting head is directly used as the target image. The target image is the shooting result, so that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0013] In one possible implementation, if the camera's shooting magnification is high (i.e., greater than the base magnification) and the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame, it indicates that the degree of jitter is large and re-framing is required. In one case, the second image generated by the adjusted shooting head is cropped according to the second cropping frame to obtain the target image, so that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photo-taking experience. In this case, the center point of the second cropping frame corresponds to the center point of the second image, and the size of the second cropping frame corresponds to the shooting magnification. In another case, due to the limitation of the camera position, there is an upper limit to the adjustment of the camera position. Therefore, not only is the frame re-framed to offset some of the jitter by adjusting the camera position, but on this basis, the second cropping frame is also transformed in position according to the jitter offset information to obtain a correction cropping frame; and the second image is cropped according to the correction cropping frame to obtain the target image. In this way, part of the jitter can be offset again by changing the position of the second cropping frame, achieving two jitter corrections, greatly reducing the FOV jump problem and improving the user's photo-taking experience.

[0014] In one possible implementation, when the camera's shooting magnification is high (i.e., greater than the base magnification), cropping is required. Based on this, the first cropping frame is repositioned according to the shake offset information to obtain a target cropping frame. The target image is then determined by determining the positional relationship between the target cropping frame and the first image, combining the target cropping frame and the first image. This ensures that the final target image corresponds to the preview image, reducing FOV jumps and improving the user's photography experience.

[0015] In one possible implementation, when the first image includes a target cropping frame, it indicates that the degree of jitter is small, indicating that the first image can cover the offset caused by the jitter of the terminal device. The target cropping frame is used to crop the first image to obtain the target image without reframing, thereby improving the photography efficiency. When the first image does not include a target cropping frame, or when there is an intersection between the first image and the target cropping frame, it indicates that the degree of jitter is large, but there is still no need to reframe. Part of the boundary of the target cropping frame is aligned with the boundary of the first image to obtain an aligned cropping frame; cropping the first image according to the aligned cropping frame to obtain the target image can also offset part of the jitter, improve the target image, reduce the FOV jump problem, and improve the user's photography experience.

[0016] In one possible implementation, the jitter offset information can be obtained in the following manner. During the shutter delay time (from the moment the capture operation is received to the moment the first image is framed), the optical image stabilizer collects multiple camera position offset information. Based on the multiple position offset information, the overall offset degree of the camera at the moment of frame generation (i.e., the jitter offset information) is calculated. The first image is then corrected based on the jitter offset information to obtain a target image. This ensures that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photography experience.

[0017] In one possible implementation, the shooting method can be applied to various scenarios of non-ZSL framing strategies, in which the camera frames in non-ZSL mode. Among them, long exposure mode, night scene mode, medium and low brightness scenes (for example, the ambient brightness is lower than the preset brightness threshold), etc. adopt non-ZSL framing strategies. Cameras with larger shooting magnifications usually do not support ZSL image output and need to adopt non-ZSL framing strategies. When there is a difference in the processing flow of the preview stream and the photo stream, the preview stream and the photo stream are two different sets of processing flows and cannot be framed immediately.

[0018] The embodiment of the present application aims at various scenarios of non-ZSL frame output strategies, and corrects the first image through statistical jitter offset information, so that the final corrected target image corresponds to the preview image, reduces the FOV jump problem, and improves the user's photo-taking experience.

[0019] In one possible implementation, a coordinate system is established with the center point of the first image, and the coordinates of the four boundaries of the first cropping frame are determined according to the size of the first cropping frame. The four boundaries are coordinate-mapped according to the jitter offset information, and the positions of the four boundaries in the coordinate system (i.e., the positions of the four boundaries of the target cropping frame) can be obtained. In this way, the positions of the boundaries of the first image in the coordinate system can be determined according to the size of the first image, and the positional relationship between the four boundaries and the boundaries of the first image can be compared in combination with the positions of the four boundaries in the coordinate system. When all boundaries of the target cropping frame do not exceed the boundaries of the first image, it indicates that the first image contains the target cropping frame. When any boundary of the target cropping frame exceeds the boundary of the first image, it indicates that there is an intersection between the first image and the target cropping frame. If all boundaries of the target cropping frame exceed the boundary of the first image, it indicates that the first image does not contain the target cropping frame.

[0020] In one possible implementation, a coordinate system is established with the center point of the first image, and the coordinates of the four vertices of the first cropping frame are determined according to the size of the first cropping frame. The coordinates of the four vertices are mapped according to the jitter offset information to obtain the positions of the four vertices in the coordinate system (i.e., the coordinates of the four vertices of the target cropping frame). In this way, the area of ​​the first image in the coordinate system can be determined according to the size of the first image, and the positional relationship between the four vertices and the area of ​​the first image can be compared in combination with the positions of the four vertices in the coordinate system. When all the vertices of the target cropping frame are located in the area formed by the boundary of the first image, it is indicated that the first image contains the target cropping frame. When any vertex of the target cropping frame is not located in the area formed by the boundary of the first image, it is indicated that there is an intersection between the first image and the target cropping frame. When all the vertices of the target cropping frame are not located in the area formed by the boundary of the first image, it is indicated that the first image does not contain the target cropping frame.

[0021] In the second aspect, a terminal device is provided, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the computer program is executed by the processor, the shooting method involved in the first aspect or any one of the implementation methods of the first aspect is implemented.

[0022] In a third aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer executes the shooting method involved in the first aspect or any one of the implementations of the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a chip system, which can be applied to a terminal device. The chip system includes one or more processors, which are used to call computer instructions to enable the terminal device to execute the shooting method described in the first aspect or any one of the implementation methods of the first aspect.

[0024] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the shooting method described in the first aspect or any one of the implementations of the first aspect.

[0025] The technical effects obtained by the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of FOV jump during non-ZSL shooting provided by an embodiment of the present application;

[0027] Figure 2 This is a schematic diagram of another FOV jump during non-ZSL shooting provided by an embodiment of the present application;

[0028] Figure 3 This is a schematic diagram of a zsl frame outgoing strategy provided in an embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of a ZSL frame outgoing process provided by an embodiment of the present application;

[0030] Figure 5 This is a schematic diagram of another ZSL framing process provided by an embodiment of the present application;

[0031] Figure 6 This is a schematic diagram of a non-ZSL frame outgoing strategy provided by an embodiment of the present application;

[0032] Figure 7 This is a schematic diagram of a non-ZSL frame outgoing process provided by an embodiment of the present application;

[0033] Figure 8 This is a schematic diagram of another non-ZSL framing process provided by an embodiment of the present application;

[0034] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0035] Figure 10 This is a block diagram of a software system of a terminal device provided in an embodiment of the present application;

[0036] Figure 11This is a schematic diagram of multiple position offset information provided by an embodiment of the present application;

[0037] Figure 12 is a schematic diagram of a relationship between a target cropping frame and a first image provided by an embodiment of the present application;

[0038] Figure 13 This is a schematic diagram of a frame output process when taking photos at a high magnification ratio under non-ZSL conditions provided by an embodiment of the present application;

[0039] Figure 14 is a schematic diagram of a relationship between an alignment cropping frame and a first image provided by an embodiment of the present application;

[0040] Figure 15 It is a flowchart of a shooting method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0042] It should be understood that the “multiple” mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, words such as “first” and “second” are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as “first” and “second” do not limit the quantity and execution order, and words such as “first” and “second” do not necessarily limit them to be different.

[0043] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0044] The shooting method provided in the embodiments of the present application can be applied to terminal devices, including but not limited to mobile phones, mobile phones, foldable phones, smart wearable devices, tablet computers (portable Android devices, PADs), handheld computers, netbooks, laptops, desktop computers, portable computers, handheld computers, wireless terminal devices, smart home devices, communication devices, digital broadcast receivers, personal digital assistants (PDAs), portable multimedia players (PMPs), ultra-mobile personal computers (UMPCs), virtual reality devices, and other electronic devices with shooting functions. The embodiments of the present application do not limit the specific type of the terminal device.

[0045] The shooting method provided in the embodiments of the present application can be applied to various scenarios in which shooting is performed using a non-ZSL framing strategy. In this scenario, the camera frames in a non-ZSL mode, and the non-ZSL mode is any of the following: long exposure mode, night scene mode, the ambient brightness is lower than a preset brightness threshold, the camera does not support zero-second delay, and there are differences in the processing procedures of the preview stream and the photo stream.

[0046] For example, in long-exposure mode and low- to medium-light scenes (for example, when the ambient brightness is below a preset brightness threshold), the shutter time needs to be extended to capture more light when taking photos. Therefore, the camera needs to adopt a non-ZSL framing strategy. In night scene mode, long exposure is required to improve image quality. In this case, the camera will adopt a non-ZSL framing strategy. In scenarios where a better signal-to-noise ratio is required (for example, the image signal-to-noise ratio is greater than the preset signal-to-noise ratio), the preview stream and the photo stream need to use different processing flows. In this case, the camera will adopt a non-ZSL framing strategy. Cameras with larger shooting magnifications generally do not support ZSL framing. In such scenarios, the camera needs to adopt a non-ZSL framing strategy. Some cameras have two different processing flows for the preview stream and the photo stream, and cannot produce frames immediately. In such scenarios, the camera needs to adopt a non-ZSL framing strategy.

[0047] The preset brightness threshold may be appropriately set by those skilled in the art according to actual conditions, as long as it can indicate that the current ambient brightness is low.

[0048] Take the mobile phone as an example. The camera application of the mobile phone has one or more shooting modes such as night scene, portrait, photo, video, panorama, slow motion or time-lapse photography. Usually, the night scene mode requires a long exposure time to improve the image quality. At this time, the camera will use the non-zsl frame strategy. Figure 1 Provide explanation.

[0049] For example, Figure 1 As shown, Figure 1 This is a schematic diagram of FOV jump during non-ZSL shooting provided by an embodiment of the present application. Figure 1 As shown in Figure A, the user can open the camera application by clicking the camera application icon. After detecting that the user has opened the camera application, the mobile phone can call the camera application to enter a certain shooting mode and open the camera to display the shooting preview interface in the shooting mode. The shooting preview interface includes a preview window, and the preview window displays the preview image before shooting in real time, such as Figure 1 As shown in Figure B. The shooting preview interface can also include a magnification option (or zoom option), and the user can select the current shooting magnification in the magnification option. Figure 1 Figure B in the figure is an example of a night scene mode with a shooting magnification of 1.0 (i.e., ×1). The main camera needs to use a non-zsl frame output strategy in night scene mode. When the user clicks the photo control, the main camera uses a non-zsl frame output strategy to output a frame, which takes about 200ms-300ms. If the user shakes the phone during this process, the output target image will be inconsistent with the preview image when the photo control is clicked. Figure 1 As shown in Figure C. Figure 1 The target image shown in Figure C is Figure 1 The preview image shown in Figure B is inconsistent, and FOV jumps occur.

[0050] In view of the above problems, the embodiment of the present application provides a shooting method. Through the shooting method provided by the embodiment of the present application, the camera's jitter offset information from the time when the user clicks the photo control to the time when the target image is generated can be obtained, and the camera's jitter offset information can be used to obtain the jitter offset information. Figure 1 Correct the C image in the image so that the final corrected output image is consistent with the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0051] With the development of electronic technology, mobile phones can be equipped with cameras of different focal lengths, which can achieve zoom shooting with a larger magnification range (for example, 1.0x shooting magnification to 20x shooting magnification), meeting the user's various shooting needs. Some cameras with larger shooting magnifications need to adopt a non-zsl framing strategy. For example, a telephoto camera can have a shooting magnification from 3.0 to 10.0. Basic magnification framing is to directly use a non-zsl framing strategy to output the target image, where the basic magnification can also be called the default magnification or the original magnification or the minimum magnification. High magnification framing is to crop (also called cropping) the image corresponding to the basic magnification to obtain the target image. In the above example, 3.0 is the basic magnification of the telephoto camera. When framing with a shooting magnification of 3.0, the target image is directly output; when framing with a shooting magnification of 5.0, it is cropped on the image corresponding to 3.0 to obtain the target image. The following is through Figure 2 Provide explanation.

[0052] For example, Figure 2 As shown, Figure 2 This is another schematic diagram of FOV jump during non-ZSL shooting provided by the embodiment of the present application. Figure 2 As shown in Figure A, the user can open the camera application by clicking the camera application icon. The phone displays the shooting preview interface, which includes a preview window. The preview window displays the preview image before taking the photo in real time. Assuming that the user chooses to zoom in and has set the current shooting magnification to 3.0 in the magnification option, as shown in the figure below: Figure 2 As shown in Figure B. The user can set the magnification option again, and the phone can magnify the image captured by the camera again. Figure 2 Figure C in the figure uses the telephoto camera in photo mode with a shooting magnification of 7.0 (i.e., ×7) as an example. When the user clicks the photo control, the telephoto camera uses a non-zsl frame output strategy, which takes about 200ms-300ms. If the user's hand shakes during this process, the output image will be inconsistent with the preview image when the photo control is clicked. Figure 2 As shown in Figure D. Figure 2 The target image shown in Figure D is Figure 2 The preview image shown in Figure C is inconsistent, with the FOV jumping. Generally speaking, hand shake at high magnifications has a greater impact on the results than at basic magnifications.

[0053] In view of the above problems, the embodiment of the present application provides a shooting method, through which the shooting method provided by the embodiment of the present application obtains the camera's jitter offset information from the time when the user clicks the photo control to the time when the target image is generated, and the camera is adjusted according to the jitter offset information. Figure 2The D image in the image is corrected so that the final corrected output image is consistent with the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0054] It should be noted that the magnification option can also be hidden in the shooting preview interface, or the mobile phone can adjust the current magnification accordingly according to the user's pinching operation in the preview window. The embodiment of the present application does not impose any restrictions on this. Figure 1 and Figure 2 The example description is based on the mobile phone capturing the captured image in the preview scenario of the camera mode. It can be understood that the mobile phone can also capture the captured image in other shooting modes (for example, video, panorama, slow motion or time-lapse photography, etc.), and the embodiments of the present application do not impose any restrictions on this.

[0055] The following Figure 3 、 Figure 4 and Figure 5 Describe the zsl frame strategy, through Figure 6 、 Figure 7 and Figure 8 Describes the non-ZSL frame outgoing strategy.

[0056] For example, Figure 3 As shown, Figure 3 This is a schematic diagram of a zsl frame strategy provided by an embodiment of the present application. The preview stream includes multiple preview images in the preview screen. The nth moment is the moment of clicking to take a photo. The preview image is F n Frame. The photo stream corresponds to the target image and outputs the photo frame, i.e. F n Frame, preview image and output photo frame are the same frame, there is no FOV jump, this is the ZSL frame strategy.

[0057] In one embodiment, the zsl framing strategy may include two modes: single shot and burst mode.

[0058] For example, Figure 4 As shown, Figure 4 This is a schematic diagram of a zsl frame outgoing process provided by an embodiment of the present application. Figure 4The figure shows the framing process of a single shot. After previewing in the preview screen, the sensor and video front-end (VFE) will generate snapshots and preview frames, and will retain the latest frames in the image buffer module. Once the "take picture" event is triggered, the shooting system will calculate the actual shooting time, find the corresponding frame from the image buffer as soon as possible, and return it to the user. Frame 1 (frame 1), frame 3 (frame 3), frame 5 (frame5), frame 7 (frame 7), frame 9 (frame 9), and frame 11 (frame 11) are preview frames that change over time. The user clicks to take a picture at frame 3, and the shutter captures frame 7. Shutter lag refers to the time from pressing the shutter button to the start of exposure, that is, from the moment you click to take a picture to the moment the frame is released. By calculating the shutter delay time, a certain frame is identified as the frame when the photo was taken. This is equivalent to finding the frame when the shutter was pressed, that is, frame 3. Frame 3 is retrieved from the image buffer through playback and returned to the user.

[0059] For example, Figure 5 As shown, Figure 5 This is a schematic diagram of another ZSL framing process provided by an embodiment of the present application; Figure 4 The figure shows the frame-out process in burst mode. Burst mode is a natural extension of the single shot feature. With this function, the shutter captures not only the current frame, but also several frames before and after the current frame, saving the sequence of captured frames to memory. Users can be provided with different snapshot times from which to extract one or more frames. The frames selected for extraction can include retroactive frames and future frames, where retroactive frames and future frames are distinguished relative to the actual shutter time. Frames 1, 3, 5, 7, 9, and 11 are preview frames that change over time. The shutter captures frame 7. Frames 5 (i.e., the retroactive frame), 7, and 9 (i.e., the future frame) are retrieved from memory and returned to the user through playback.

[0060] For example, Figure 6 As shown, Figure 6 This is a schematic diagram of a non-zsl frame strategy provided by an embodiment of the present application. The preview stream includes multiple preview images in the preview screen. The nth moment is the moment of clicking to take a photo. The preview image is F nThe photo stream corresponds to the target image. Due to the time taken to re-output the frame (that is, the time taken to output the frame), which is about 200ms-300ms, the output photo frame is not F n frame, but F m Frame, preview image and output photo frame are inconsistent, there is FOV jump, this is the non-ZSL frame strategy.

[0061] The following describes the non-zsl frame process. Figure 7 As shown, Figure 7 : This is a schematic diagram of a non-ZSL framing process provided by an embodiment of the present application. The video front end (VFE) includes a viewfinder and a snapshot module, and a preview frame is shown in the shooting preview screen. After pressing the shutter, take a picture. The non-ZSL framing process includes automatic focus (AF) and prepare snapshots. Since the output image formats of the preview stream and the photo stream are different, it is necessary to reconfigure the sensor and video front end (re-config sensor & VFE), output a thumbnail, and expose and read out the original image (yuvraw data) corresponding to the thumbnail. After the raw image is called back (raw callback), image encoding (jpeg encoding) is performed, the original image in yuv format is encoded into JPEG format, and the callback image (jpegcallback), that is, the target image, is output. Then restart the preview. Shutter lag includes the time required for AF and snapshot preparation, sensor and VFE reconfiguration, exposure, and readout. YUV is a color encoding method, and JPEG is an image format.

[0062] For example, Figure 8 As shown, Figure 8This is a schematic diagram of another non-ZSL framing process provided by an embodiment of the present application. The sensor transmits the original image to the image signal processor (ISP) through the image transmission interface (IF). The ISP preprocesses the original image, for example, removing noise, lens correction, bad pixel correction, color interpolation, color correction, etc., and then transmits the preprocessed image to the video front end (VFE). The VFE converts the image into a data format and then transmits the processed image (including thumbnails and main images) to the post-processing module (C2D). After the thumbnails and main images are further processed by C2D, they are transmitted to the hardware abstraction layer (HAL). After image format conversion, the target image is obtained, and then the target image is transmitted to the camera application. Thumbnails can be displayed in the shooting preview screen of the camera application.

[0063] above Figure 6-Figure 8 The figure shows a traditional snapshot, which includes a preview stream and a photo stream. When the preview starts, a preview frame is output. When taking a snapshot, pressing the shutter button starts taking the photo, and the preview stream stops. When taking a photo, a series of actions are performed, including autofocus (AF), auto exposure (AE), and preparing to take the photo. The photo stream calls back the original image and then starts encoding the original image into an image format (JPEG) for storage. After the photo is taken, the preview stream output starts again. In a traditional snapshot, that is, when taking a photo, the preview stops. After the photo is taken, the preview image can be redisplayed on the shooting preview screen after the post-processing module (C2D) completes the processing of the main image and thumbnail.

[0064] above Figure 6-Figure 8 The non-ZSL framing strategy is shown. It can be seen that the non-ZSL framing strategy is affected by automatic focus (AF) and auto exposure (AE), requiring re-framing. Therefore, for non-ZSL framing, shutter lag can easily cause FOV jumps.

[0065] Based on the above-mentioned problem of image inconsistency in non-ZSL scenarios, an embodiment of the present application provides a shooting method. In a non-ZSL scenario, the jitter offset information of the camera in the terminal device is obtained from the moment the shooting button is clicked (i.e., the shutter is pressed) to the frame-out moment. The jitter offset information reflects the degree of jitter of the camera within the shutter delay time (from the moment the shutter is pressed to the frame-out moment). The image at the frame-out moment is corrected according to the jitter offset information. On the one hand, when the camera is shooting at its basic magnification, the position of the camera can be offset (also called bias) according to the jitter offset information and then re-framed. The re-framed image corresponds to the preview image in the shooting preview screen, thereby achieving frame correction. On the other hand, in the case of high-magnification shooting (i.e., zoomed shooting) and when the jitter is not severe, the cropping frame (also called cropping frame) of the image at the frame-out moment can be offset according to the jitter offset information and then cropped on the image at the frame-out moment and output, thereby achieving frame correction. On the other hand, when shooting at high magnification (i.e., zoomed-in shooting) and with severe shaking, the camera position can be offset according to the shaking offset information and then reframed, and the cropping frame of the reframed image can be offset according to the shaking offset information, and then the image can be cropped and output on the reframed image to achieve frame correction, so that the final corrected output image is consistent with the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0066] Before explaining in detail the shooting method provided in the embodiment of the present application, the terminal device involved in the embodiment of the present application is first explained.

[0067] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 9The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0068] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0069] The processor 110 may include one or more processing units, for example, an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0070] The controller may be the nerve center and command center of the terminal device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0071] The processor 110 may also include a memory for storing instructions and data.

[0072] In some embodiments, the processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0073] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt a different interface connection method from the above embodiments, or a combination of multiple interface connection methods.

[0074] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0075] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that terminal device 100 can communicate with the network and other devices through wireless communication technology.

[0076] The terminal device 100 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0077] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is an integer greater than 1.

[0078] The terminal device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0079] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0080] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is an integer greater than 1.

[0081] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. This allows terminal device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0082] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0083] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created by the terminal device 100 during use (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0084] The terminal device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D and the application processor.

[0085] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, terminal device 100 detects the touch intensity based on pressure sensor 180A. Terminal device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0086] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse motion to achieve anti-shake.

[0087] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 100, at a different location from the display screen 194.

[0088] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the terminal device 100 by inserting or removing it from the SIM card interface 195. The terminal device 100 can support one or N SIM card interfaces, where N is an integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0089] Next, the software system of the terminal device 100 will be described.

[0090] The software system of the terminal device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to exemplify the software system of the terminal device 100.

[0091] Figure 10 This is a block diagram of a software system of a terminal device 100 provided in an embodiment of the present application. Figure 10 The layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system layer, and the kernel layer.

[0092] The application layer can include a series of application packages. Figure 10 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0093] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 10 As shown, the application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, and notification manager. The window manager manages window programs. It can obtain the display size, determine whether a status bar is present, lock the screen, and take screenshots. The content provider stores and retrieves data and makes it accessible to applications. This data may include video, images, audio, incoming and outgoing calls, browsing history and bookmarks, and the phone book. The view system includes visual controls, such as those for displaying text and images. The view system is used to construct the application's display interface, which may consist of one or more views, such as a view that displays a text message notification icon, a view that displays text, and a view that displays images. The telephony manager provides communication functions for the terminal device 100, such as managing call status (including connected and disconnected calls). The resource manager provides various resources to applications, such as localized strings, icons, images, layout files, and video files. The notification manager enables applications to display notifications in the status bar. These notifications can be used to convey informational messages and can disappear automatically after a short pause, without requiring user interaction. For example, a notification manager is used to notify users of completed downloads, message alerts, and more. A notification manager can also appear as an icon or scrolling text bar in the system's top status bar, such as notifications from background applications. A notification manager can also appear as a dialog window on the screen, such as a text message in the status bar, a beep, a vibration on an electronic device, or a flashing indicator light.

[0094] The Android Runtime consists of core libraries and a virtual machine (VM). The Android Runtime is responsible for scheduling and management of the Android system. The core libraries consist of two parts: one containing the Java language's callable functions and the other the Android core library. The application layer and the application framework layer run in the VM. The VM executes the Java files in the application layer and application framework layer as binary files. The VM is responsible for performing functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0095] The system library can include multiple functional modules, such as: surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0096] The system library also includes an algorithm engine, which is used to obtain a set of position offset information of the camera in the terminal device detected by the gyroscope or rotation vector sensor, as well as the preview image included in the preview stream and the first image and second image included in the photo stream during the photo shooting process, and execute the shooting method provided in the embodiment of the present application based on the above-obtained data and images.

[0097] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0098] In this application, by calling the hardware interface in the application framework layer, the connection between the application layer above the application framework layer, the system library below the application framework layer, the kernel layer and the hardware layer can be realized to realize sensor data collection and processing.

[0099] The following describes the workflow of the software and hardware of the terminal device 100 in combination with the capture and photo shooting scene.

[0100] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the raw input event. For example, if the touch operation is a single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then calls the kernel layer to start the camera driver to capture a still image or video through the camera 193.

[0101] The following describes the jitter offset information of the camera in the terminal device.

[0102] Typically, handheld devices experience some jitter when taking photos, affecting the clarity of the captured image. Currently, optical image stabilizers (OIS) can use a gyroscope or rotation vector sensor, a device used to detect the relative position between the current and previous moments, to collect camera position offset information (also known as OIS information). Using an optical image stabilization algorithm, the camera is offset to compensate for the deviation in the light path caused by jitter, thereby ensuring effective image stabilization.

[0103] It should be noted that current technologies use OIS information for anti-shake, and anti-shake correction is performed on the current frame. However, the present embodiment uses OIS information to calculate the overall OIS offset information during the shutter delay time, thereby obtaining the camera's jitter offset information. This jitter offset information is used as correction information to correct the FOV inconsistency problem of the preview image on the terminal device.

[0104] In one implementation, OIS can be set on the camera, so that the position offset information detected by OIS itself can represent the position offset information of the camera.

[0105] In some embodiments, the terminal device may also collect position offset information of the camera in the terminal device through an electronic image stabilization (EIS) device, which is not limited in the embodiments of the present application.

[0106] Based on this, in an embodiment of the present application, position offset information is collected during the shutter delay period. The position offset information indicates the difference between the camera's position at the current sampling moment and the camera's position at the moment before the current sampling moment. There can be one or more pieces of position offset information during the shutter delay period, and based on the one or more pieces of position offset information, the jitter offset information of the camera in the terminal device can be determined.

[0107] When obtaining the jitter offset information of the camera in the terminal device, it can be achieved in the following manner: obtaining multiple position offset information of the camera collected by an optical image stabilizer or an electronic imaging device (EIS) during the period from the moment the shooting operation is received to the moment the first image is framed; and determining the jitter offset information based on the multiple position offset information. In other words, the jitter offset information is the overall offset information of the camera during the period from the moment the shooting operation is received to the moment the first image is framed.

[0108] In this embodiment of the present application, the period between the moment a capture operation is received and the moment the first image is framed is the shutter delay time. During the shutter delay time, the camera's position offset information is sampled at a preset sampling frequency to obtain multiple pieces of position offset information. Any one piece of the multiple pieces of position offset information, the sum of any multiple pieces of position offset information, or the sum of all pieces of position offset information is used as the jitter offset information.

[0109] The preset sampling frequency is the sampling frequency of the detection device. The detection device can be a gyroscope, a rotation vector sensor, a game rotation vector sensor (game rotation vector), or other sensors that are a fusion of an acceleration sensor and a gyroscope. As long as the position offset information between the current moment and the previous moment of the camera can be detected, the embodiments of the present application do not impose any restrictions on this.

[0110] As an example, the detection device samples the position offset information of the camera according to a preset sampling frequency to obtain a position offset information set; multiple position offset information within the shutter delay time is filtered out from the position offset information set; the number of multiple position offset information is determined based on the shutter delay time and the preset sampling frequency.

[0111] The position offset information can be a two-dimensional vector, and each position offset information includes a horizontal coordinate offset and a vertical coordinate offset; the horizontal coordinate offsets of multiple position offset information are added to obtain the overall horizontal coordinate; the vertical coordinate offsets of multiple position offset information are added to obtain the overall vertical coordinate; the overall horizontal coordinate and the overall vertical coordinate are used as jitter offset information.

[0112] The position offset information can also be a three-dimensional vector, each position offset information includes an x-axis offset, a y-axis offset and a z-axis offset; the x-axis offsets of multiple position offset information are added to obtain the overall x-axis coordinate; the y-axis offsets of multiple position offset information are added to obtain the overall y-axis coordinate; the z-axis offsets of multiple position offset information are added to obtain the overall z-axis coordinate; the overall x-axis coordinate, the overall y-axis coordinate and the overall z-axis coordinate are used as jitter offset information.

[0113] For example, taking a photo-taking scene as an example, Figure 11 is a schematic diagram of multiple position offset information provided by an embodiment of the present application. Figure 11 The figure shows the four position offset information of the camera within the shutter delay time, and takes the position offset information of the OIS itself (ie, the position offset information of the camera) as an example for explanation. Figure 11 Where t represents the sampling time. t = 0 represents the time when the shooting operation is received or the time when the photo control is clicked. The initial position of ois at t = 0 is obtained, that is, the initial position of the camera when the photo control is clicked. Continue to obtain the position of ois according to the preset sampling frequency, obtain the camera movement position 1 at t = 1, obtain the camera movement position 2 at t = 2, and so on, until t = n, t = n represents the frame time, and obtain the camera movement position n at t = n. Add up the n position offset information to obtain the jitter offset information. Taking the position offset information as a two-dimensional vector as an example, the horizontal and vertical coordinates of each position offset information are accumulated separately to obtain the jitter offset information. As shown in formula (1).

[0114] Offset(x,y)=f(x1,y1)+f(x2,y2)+…f(xn,yn) (1)

[0115] In the above formula (3), x represents the horizontal coordinate, y represents the vertical coordinate, n represents the sampling time, and f represents the position offset information. f(x1, y1) represents the offset between the camera position at t = 1 and the camera position at t = 0; f(x2, y2) represents the offset between the camera position at t = 2 and the camera position at t = 1; and f(xn, yn) represents the offset between the camera position at t = n and the camera position at t = n-1. Adding the n position offset information together yields the jitter offset information offset.

[0116] In an embodiment of the present application, when the photo control is clicked (i.e., a shooting operation is received), the position offset information of each sampling moment ois is recorded until the frame is output, and multiple position offset information is obtained. The overall offset degree of the camera at the frame output moment (i.e., jitter offset information) is calculated based on the multiple position offset information. Then, the first image is corrected according to the jitter offset information to obtain a target image, so that the final target image corresponds to the preview image, reducing the fov jump problem and improving the user's photo experience. Among them, the first image is an image generated by the camera using a non-zsl frame output strategy.

[0117] It should be noted that the first image and the second image are images generated by a shooting module including a camera through a series of processing. In order to simplify the description here, the first image and the second image are described as images generated by the camera, that is, images generated by the camera.

[0118] The following describes how to shoot at the basic magnification.

[0119] The shooting method provided in the embodiment of the present application is applied to the same camera of the terminal device, and the camera adopts a non-ZSL framing strategy. The camera has a pre-set shooting magnification, and the minimum magnification is the basic magnification. When shooting with the basic magnification, the image at the moment of framing is also the first image, and there is no need to crop. When the shooting magnification is greater than the basic magnification, the image at the moment of framing is obtained after cropping (cropping) the first image. The cropping size (including the length and width of the cropping frame) is inversely correlated with the shooting magnification. The larger the shooting magnification, the smaller the cropping size.

[0120] Under a non-ZSL framing strategy, the camera in the terminal device shoots at a base magnification, and the preview stream displays a preview image to the user. In response to the user's shooting operation, a shutter delay time is experienced, and a first image is output from the shooting stream. The first image is inconsistent with the preview image. Here, the camera's jitter offset information within the shutter delay time is obtained, and the camera's position is controlled to be adjusted to a position opposite to the jitter offset information, so that the adjusted camera position corresponds to the camera's position when the shooting operation is received. For example, taking the jitter offset information as a two-dimensional vector (3, -4) as an example, the camera's position offset is controlled to be (-3, 4), and the camera's position deviation is corrected by moving the camera's position in a direction opposite to the jitter direction.

[0121] Combined with the above Figure 6 When the shooting operation is received, the shooting preview screen displays F n Frame, the image at the time of frame output is F m Frame, that is, the first image is F m Frame. Adjust the position of the camera according to the jitter offset information so that the adjusted camera position is consistent with F n The position of the camera corresponding to the frame corresponds to the position of the camera. In this way, the second image generated by the adjusted camera is F m+1 Frame, F m+1 Frame and F n Frame corresponding, F m+1 Frame is understood as F n Frame. m+1 Frame as the target image to achieve F n Frame correction. The camera outputs the photo stream F m+1 Frame and F m Frames are adjacent frames, F m+1 Frame is F m The next frame of the frame.

[0122] The following describes how to shoot at high magnification.

[0123] Under a non-ZSL framing strategy, the terminal device's camera captures at a high magnification (greater than the base magnification, or not the base magnification). The preview stream displays a preview image to the user (the preview image is a cropped version of the original image). In response to the user's capture operation, after a shutter delay, the first image is output from the capture stream. The final image is then cropped from the first image, which is inconsistent with the preview image. Here, the positional relationship between the first image and the target cropping frame is explained.

[0124] When framing at a high magnification, the first cropping frame used for cropping the first image is centered on the center point of the first image. The size of the first cropping frame is inversely correlated with the shooting magnification. The position of the first cropping frame is transformed based on the shake offset information, that is, the first cropping frame is moved in the direction opposite to the shake offset information. For example, if the shake offset information is a two-dimensional vector (-2, 3), the first cropping frame is offset by (2, -3). By moving the position of the first cropping frame in the direction opposite to the shake direction, the position deviation of the first cropping frame is corrected, thereby obtaining the target cropping frame. The target cropping frame has the same size as the first frame but a different position.

[0125] The relationship between the target cropping frame and the first image includes the following three types: the first image contains the target cropping frame, such as Figure 12 As shown in Figure A in ; there is an intersection between the first image and the target cropping frame, such as Figure 12 As shown in Figure B in ; the first image does not contain the target cropping frame, such as Figure 12 As shown in Figure C in the embodiment of the present application, the positional relationship between the target cropping frame and the first image can be determined in the following two ways.

[0126] Example 1: In a first image, coordinate mapping is performed on the four boundaries of a first cropping frame according to jitter offset information to obtain the positions of the four boundaries of a target cropping frame; if all boundaries of the target cropping frame do not exceed the boundaries of the first image, it is determined that the first image contains the target cropping frame; if any boundary of the target cropping frame exceeds the boundary of the first image, it is determined that the first image does not contain the target cropping frame, or that there is an intersection between the first image and the target cropping frame.

[0127] In the embodiment of the present application, a coordinate system is established with the center point of the first image, and the coordinates of the four boundaries of the first cropping frame are determined according to the size of the first cropping frame. The coordinates of the four boundaries are mapped according to the jitter offset information, and the positions of the four boundaries in the coordinate system can be obtained. In this way, the positions of the boundaries of the first image in the coordinate system can be determined according to the size of the first image, and the positional relationship between the four boundaries and the boundaries of the first image can be compared in combination with the positions of the four boundaries in the coordinate system. If the four boundaries do not exceed the boundaries of the first image, as shown in FIG. Figure 12As shown in Figure A, the first image contains the target cropping frame. If any of the four boundaries exceeds the boundary of the first image, such as Figure 12 As shown in Figure B, there is an intersection between the first image and the target cropping frame. If all four boundaries exceed the boundaries of the first image, such as Figure 12 As shown in Figure C, it shows that the first image does not contain the target cropping frame.

[0128] As can be seen from the positional relationship between the target cropping frame and the first image, Figure 12 Figure A in Figure 12 Figure B in Figure 12 In Figure C, the jitter degree increases successively.

[0129] In an embodiment of the present application, a coordinate system is established for the plane where the first image and the target cropping frame are located, and the positional relationship of the boundaries between the two is compared to determine the positional relationship between the target cropping frame and the first image. This eliminates the need for absolute position information of the first image and the target cropping frame, thereby improving judgment efficiency.

[0130] Example 2: In the first image, coordinates of the four vertices of the first cropping frame are mapped according to the jitter offset information to obtain the coordinates of the four vertices of the target cropping frame; when all vertices of the target cropping frame are located in the area formed by the boundary of the first image, it is determined that the first image contains the target cropping frame; when any vertex of the target cropping frame is not located in the area formed by the boundary of the first image, it is determined that the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame.

[0131] In the embodiment of the present application, a coordinate system is established with the center point of the first image, and the coordinates of the four vertices of the first cropping frame are determined according to the size of the first cropping frame. The positions of the four vertices in the coordinate system can be obtained by mapping the coordinates of the four vertices according to the jitter offset information. In this way, the area range of the first image in the coordinate system can be determined according to the size of the first image, and the positional relationship between the four vertices and the area range of the first image can be compared in combination with the positions of the four vertices in the coordinate system. If the four vertices are within the area range, such as Figure 12 As shown in Figure A, the first image contains the target clipping frame. If any of the four vertices exceeds the region, such as Figure 12 As shown in Figure B, there is an intersection between the first image and the target cropping frame. If all four vertices are outside the region, such as Figure 12 As shown in Figure C, it shows that the first image does not contain the target cropping frame.

[0132] In an embodiment of the present application, a coordinate system is established for the plane where the first image and the target cropping frame are located, and the positional relationship between the four vertices of the target cropping frame and the area range of the first image is compared to determine the positional relationship between the target cropping frame and the first image. This eliminates the need for absolute position information of the first image and the target cropping frame, thereby improving judgment efficiency.

[0133] Based on the positional relationship between the target cropping frame and the first image, the first image can be corrected in the following ways.

[0134] In the first implementation, when the shooting magnification is high, if the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame, it means that the jitter is large and a new frame is needed. Then, the camera's jitter offset information within the shutter delay time is obtained, and the camera's position is controlled to be adjusted to a position opposite to the jitter offset information, so that the adjusted camera position corresponds to the camera's position when the shooting operation is received. Figure 6 , the second image generated by the adjusted camera is F m+1 Frame, you can m+1 Frame is understood as F n Frame. According to the second cropping frame, F m+1 The frame is cut to obtain the target image to achieve F n Frame correction reduces FOV jump problems and improves the user's photo-taking experience.

[0135] The second cropping frame is centered on the center point of the second image, and the size of the second cropping frame is inversely correlated with the shooting magnification. The size of the second cropping frame is the same as that of the first cropping frame.

[0136] In the second implementation, based on the first implementation, after obtaining F m+1 After the frame, the second cropping frame is transformed according to the jitter offset information, that is, the second cropping frame is moved in the opposite direction of the jitter offset information to obtain the corrected cropping frame. m+1 The target image is obtained by cropping the frame to achieve F nFrame correction. Considering that the position of the camera needs to be adjusted and the frame needs to be re-produced, that is, the degree of jitter is relatively large, and due to the position of the camera, there is an upper limit to the adjustment of the camera position. Therefore, adjusting the position of the camera can offset part of the jitter, but cannot completely eliminate the jitter. On this basis, the offset of the second cropping frame can be used to obtain a correction cropping frame to offset part of the jitter again, reduce the fov jump problem, and improve the user's photo-taking experience. Compared with the first implementation method, the second implementation method achieves two jitter corrections by adjusting the position of the camera and the offset of the second cropping frame, which greatly reduces the fov jump problem and improves the user's photo-taking experience.

[0137] In the third implementation, when the shooting magnification is high, no re-frame is generated, and the first image generated by the camera is F m Frame, the first cropping frame is transformed according to the jitter offset information, that is, the first cropping frame is moved in the opposite direction of the jitter offset information to obtain the target cropping frame. m The following two examples illustrate this.

[0138] Example 1: If the first image contains the target cropping frame, it means that the jitter is small, which means that the first image can cover the offset caused by the jitter of the terminal device, and there is no need to re-frame. m The frame is cut to obtain the target image to achieve F n Frame correction. In this example, when the jitter is small, there is no need to re-frame, which improves shooting efficiency, reduces FOV jump problems, and improves the user's shooting experience.

[0139] For example, Figure 13 As shown, Figure 13 This is a schematic diagram of a frame-generating process when taking a high-magnification photo under non-ZSL conditions provided by an embodiment of the present application. Taking the first image including the target cropping frame as an example, the first image should be cropped with the target cropping frame due to jitter occurring during the shutter delay time. Figure 13 As shown in Figure A of FIG, in the related art, the first image is directly cropped according to the first cropping frame, resulting in inconsistency between the final target image and the preview image. In the embodiment of the present application, the first cropping frame is offset according to the jitter offset information to obtain the target cropping frame, such as Figure 13 As shown in Figure B, cropping is performed on the first image according to the target cropping frame, so that the final target image is consistent with the preview image, reducing the FOV jump problem and improving the user's photo-taking experience.

[0140] Example 2: If the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame, it means that the degree of jitter is large, but the frame is not re-produced. Align part of the boundary of the target cropping frame with the boundary of the first image, that is, move the target cropping frame in the same direction as the jitter offset information until it is aligned with the boundary of the first image, or move the first cropping frame in the opposite direction of the jitter offset information until it is aligned with the boundary of the first image, to obtain an aligned cropping frame. Figure 12 Figure B and Figure 12 Based on Figure C in Figure 14 As shown, Figure 14 is a schematic diagram of the relationship between an alignment cropping frame and a first image provided by an embodiment of the present application, Figure 14 It can be seen that the alignment cropping frame is located at the edge of the first image, that is, part of the boundary of the alignment cropping frame is located on the boundary of the first image. The purpose is to make the alignment cropping frame fall within the first image so that the target image can be obtained by cropping in the first image according to the alignment cropping frame. In this example, although the degree of jitter is large, the frame is not re-produced. By moving the target cropping frame or the first cropping frame, moving it to the first image, and using the final alignment cropping frame for cropping in the first image, part of the jitter can be offset, the target image can be improved, the FOV jump problem can be reduced, and the user's photo-taking experience can be improved.

[0141] In the above three implementations, it can be seen that when a frame needs to be re-produced, the position of the camera is controlled to be adjusted to a position opposite to the jitter offset information, and then the second image (F m+1 The target image is determined by re-framing the frame. The conditions for re-framing (i.e., the first preset conditions) may include the camera's shooting magnification being the base magnification; or the camera's shooting magnification being greater than the base magnification, and the first image not containing the target cropping frame, or the first image intersecting the target cropping frame.

[0142] The condition that does not require re-framing (ie, the second preset condition) may include that the shooting magnification of the camera is greater than the basic magnification and the first image contains the target cropping frame.

[0143] For the case where the camera's shooting magnification is greater than the base magnification, and the first image does not contain the target cropping frame, or there is an intersection between the first image and the target cropping frame, you can choose to reframe or not. In the embodiment of the present application, in this case, a person skilled in the art can choose between reframing and not reframing according to the actual situation. Compared with not reframing, reframing has a better technical effect in reducing the FOV jump problem; compared with reframing, not reframing has a better technical effect in framing efficiency.

[0144] It should be noted that, when the shooting magnification is high, if the first image contains the target cropping frame, it means that the degree of jitter is small, but the frame is still re-generated. The camera's jitter offset information within the shutter delay time is obtained, and the camera's position is controlled to be adjusted to a position opposite to the jitter offset information, so that the adjusted camera position corresponds to the camera's position when the shooting operation is received. Combined with the above Figure 6 , the second image generated by the adjusted camera is F m+1 Frame. According to the second cropping frame, F m+1 The frame is cropped to obtain the target image, where the center point of the second cropping frame corresponds to F m+1 The center point of the frame, the size of the second cropping frame corresponds to the shooting magnification. In this example, the F n Frame correction reduces FOV jump problems, improves the user's photo-taking experience, and increases the diversity of shooting methods.

[0145] Based on the above Figure 11-14 , the embodiment of the present application provides a shooting method, so that the target image output after the final correction corresponds to the preview image, reducing the field of view jump problem and improving the user's shooting experience. Figure 15 As shown, Figure 15 It is a flowchart of a shooting method provided in an embodiment of the present application.

[0146] S101: Display a first interface, where the first interface includes a shooting preview screen.

[0147] The first interface can be a shooting preview interface, see the above Figure 1 and Figure 2 The shooting preview interface in the first interface can display the shooting preview screen, such as Figure 1 Schools shown in or Figure 2 Portrait displayed in .

[0148] For example, taking the terminal device as a mobile phone, as mentioned above Figure 1 or Figure 2 The user can click on the camera application icon, and the mobile phone responds to the user's operation and starts the camera application, thereby displaying the first interface.

[0149] S102: In response to a shooting operation of a user, obtain jitter offset information of a camera in a terminal device from a moment when the shooting operation is received to a moment when a first image is framed.

[0150] The user's shooting operation can be the user pressing or touching the photo control, clicking or pressing the volume button, remote control, voice control or gesture control, etc., and the embodiments of the present application do not limit this, as long as it is an operation that can realize the photo taking. For non-ZSL frame output strategies, when the photo is started, the preview stream stops, and the gyroscope or rotation vector sensor in OIS can be used to collect the jitter offset information of the camera in the terminal device within the shutter delay time (from the moment the above-mentioned shooting operation is received to the moment the first image is framed). The jitter offset information can be found in the above Figure 11 The description will not be repeated here.

[0151] S103: Determine whether the shooting magnification is the basic magnification.

[0152] If yes, execute S104; if no, execute S106.

[0153] S104: Control the position of the camera to adjust to a position opposite to the jitter offset information.

[0154] The jitter offset information can be a two-dimensional or three-dimensional vector. Using the camera's current position as the initial position, the camera is adjusted to a position opposite to the jitter offset information. For example, the jitter offset information is a two-dimensional vector (3, -4), which controls the camera's position offset to (-3, 4) to correct for camera position deviation. The jitter offset information is a three-dimensional vector (2, 5, -2), which controls the camera's position offset to (-2, -5, 2) to correct for camera position deviation.

[0155] S105: Use the second image generated by the adjusted camera head as the target image.

[0156] When the shooting magnification is the basic magnification, there is no need to crop. The second image generated by the adjusted camera head is directly used as the target image. The target image is the shooting result, so that the final target image corresponds to the preview image, reducing the FOV jump problem and improving the user's photography experience.

[0157] S106 , transforming the position of the first cropping frame according to the shake offset information to obtain a target cropping frame; the center point of the first cropping frame corresponds to the center point of the first image, and the size of the first cropping frame corresponds to the shooting magnification.

[0158] For high magnifications (i.e., not the base magnification), cropping is required. A coordinate system is established with the center point of the first image. The first cropping frame is transformed (or mapped) to a position opposite to the shake offset information to obtain the target cropping frame. The first and target cropping frames have the same size and are inversely correlated with the magnification.

[0159] S107: Determine whether the first image contains a target cropping frame.

[0160] If yes, then execute S111. If no, it means that the first image does not contain the target cropping frame or there is an intersection between the first image and the target cropping frame, then execute S1081-S1083.

[0161] Whether the first image contains the target cropping frame can be determined by the position relationship between the boundary position or the vertex and the region. Figure 12 The description in , will not be repeated here.

[0162] S1081: Control the position of the camera to be adjusted to a position opposite to the jitter offset information, and generate a second image through the adjusted camera head.

[0163] If the first image does not contain the target cropping frame or if the first image intersects the target cropping frame, this indicates a significant degree of jitter and requires re-framing. The camera position is controlled to adjust to a position opposite to the jitter offset information, such that the adjusted camera position corresponds to the camera position when the capture operation was received.

[0164] S1082: Transform the position of the second cropping frame according to the shake offset information to obtain a corrected cropping frame; the center point of the second cropping frame corresponds to the center point of the second image, and the size of the second cropping frame corresponds to the shooting magnification.

[0165] S1083: Crop the second image according to the corrected cropping frame to obtain a target image.

[0166] Due to the high degree of jitter and the limitations of the camera's position, there's an upper limit to how much jitter can be adjusted. Therefore, adjusting the camera's position can offset some of the jitter, but it can't completely eliminate it. Based on this, the second cropping frame is transformed (or mapped) to a position opposite to the jitter offset information to obtain a corrected cropping frame. This corrected cropping frame further offsets some of the jitter, reducing FOV jumps and improving the user's photography experience.

[0167] S109 , cropping the first image according to the target cropping frame to obtain a target image.

[0168] If the first image contains the target cropping frame, the jitter is minimal and no re-framing is required. The first image is cropped based on the target cropping frame to obtain the target image. This ensures that the final target image is consistent with the preview image, minimizing FOV jumps and improving the user's photography experience.

[0169] In an embodiment of the present application, after obtaining the target image, the terminal device displays a second interface, which includes the target image or a thumbnail of the target image. Taking a mobile phone as an example, after obtaining the target image, the mobile phone displays a thumbnail of the target image on the shooting preview screen. Clicking the thumbnail can further display the target image on the shooting preview screen. Taking a digital camera as an example, after obtaining the target image, the target image can be displayed on the second interface of the digital camera through image viewing operations, etc.

[0170] The embodiment of the present application aims to solve the technical problem that the preview image is inconsistent with the final target image FOV in the photo scene using a non-ZSL framing strategy, and improve the user's photo experience. Taking the terminal device as a mobile phone as an example, when the user clicks the photo control of the mobile phone (that is, receives the user's shooting operation), the current OIS information of the mobile phone (that is, the current position of the camera) is recorded, which can be set to the initial point (0, 0). Then statistics are collected from clicking the photo control to the photo algorithm framing (that is, the framing moment of the first image), at each moment, the position offset information (that is, OIS information) of the mobile phone, until the mobile phone framing is completed (that is, generating the first image). Determine whether the shooting magnification is the basic magnification. When the shooting magnification is the basic magnification, the jitter offset information is calculated based on multiple position offset information, and the camera is offset based on the jitter offset information, that is, the position of the control camera is adjusted to the position opposite to the jitter offset information, and then the adjusted camera is used to reframe to obtain the target image, which can be used as a reference frame to provide a benchmark for subsequent image processing. When the shooting magnification is high (i.e., the shooting magnification is not the basic magnification), the first image (i.e., the complete image at the basic magnification at the time of frame generation) is usually cropped. In the embodiment of the present application, the statistical jitter offset information is added as an offset during cropping, and the jitter offset information is fed into the shooting algorithm as the offset parameter of the first crop frame (crop) (the shooting algorithm can be integrated in the Figure 10 In the algorithm engine), after correcting the offset of the first cropping frame, the first image is cropped to obtain the final target image. That is, the first cropping frame that coincides with the center point of the first image is transformed to a position opposite to the jitter offset information to obtain a target cropping frame, and the first image is cropped according to the target cropping frame to obtain the target image. By counting the jitter offset information from clicking the photo control to the photo algorithm framing (i.e., the framing moment of the first image), the final cropping position is adjusted, thereby reducing the problem of obvious FOV jumps between the target image in the photo stream and the preview image when the photo control is clicked, maintaining the FOV consistency between the preview image and the final target image, and improving the user's photo experience.

[0171] The embodiment of the present application utilizes OIS information to count the overall offset information of OIS within the shutter delay time, thereby obtaining the jitter offset information of the camera, and uses the jitter offset information as correction information to correct the FOV inconsistency problem of the preview image of the terminal device.

[0172] For example, when the shooting magnification is the basic magnification, the OIS information is used to control the camera to adjust to a position opposite to the OIS information, and then the adjusted camera is used to re-frame to achieve frame offset and obtain the target image. When the shooting magnification is high, the OIS information is used as the offset information of the first cropping frame in the first image, and the first cropping frame is positionally transformed to obtain the target cropping frame, achieving the output position offset to cover the offset caused by the jitter of the terminal device, and then the first image is cropped using the target cropping frame to obtain the target image.

[0173] In the embodiment of the present application, the log data (log) during the shooting process can be captured for analysis to determine whether the position offset information of the camera (i.e., OIS information) has been counted during the shooting process, and the OIS information can be used to adjust the camera, reframe, or use the OIS information to change the position of the cropping frame. If so, it means that the shooting process adopts the shooting method provided by the embodiment of the present application. Normally, when the shooting magnification is high, the mobile phone will use non-ZSL framing. Based on this, a high magnification (for example, ×10) can be used for shooting. The object being shot remains motionless. After clicking the photo button on the mobile phone, move the mobile phone at a certain speed to see if the final shooting result is offset in the opposite direction. If an offset occurs, it means that the shooting process adopts the shooting method provided by the embodiment of the present application.

[0174] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (such as a coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0175] The above are optional embodiments provided for this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the technical scope disclosed in this application should be included in the scope of protection of this application.

Claims

1. A shooting method, characterized in that: Applied to a terminal device, the method includes: Displaying a first interface, wherein the first interface includes a shooting preview screen; In response to a user's shooting operation, obtaining jitter offset information of a camera in the terminal device between a moment when the shooting operation is received and a moment when a first image is framed; the first image is an image generated by the camera in a non-zero-second delay mode; The first image is corrected according to the jitter offset information to obtain a target image, where the target image is a shooting result.

2. The method according to claim 1, wherein Correcting the first image according to the jitter offset information to obtain a target image includes: Controlling the position of the camera to be adjusted to a position opposite to the jitter offset information, so that the adjusted position of the camera corresponds to the position of the camera when the shooting operation is received; The target image is determined based on the second image generated by the adjusted camera to achieve correction of the first image; the second image is the next frame of the first image in the photo stream output by the camera.

3. The method according to claim 2, wherein The controlling the position of the camera to adjust to a position opposite to the jitter offset information includes: When a first preset condition is met, controlling the position of the camera to adjust to a position opposite to the shake offset information; the first preset condition includes: the shooting magnification of the camera is a base magnification; or the shooting magnification of the camera is greater than the base magnification, and the first image does not include a target cropping frame, or the first image and the target cropping frame intersect; The target cropping frame is obtained by positionally transforming the first cropping frame according to the jitter offset information, the center point of the first cropping frame corresponds to the center point of the first image, and the size of the first cropping frame corresponds to the shooting magnification.

4. The method according to claim 3, wherein When the shooting magnification of the camera is a basic magnification, determining the target image according to the second image generated by the adjusted camera includes: The second image generated by the adjusted shooting head is used as the target image.

5. The method according to claim 3, wherein When the shooting magnification of the camera is greater than the base magnification and the first image does not include a target cropping frame, or the first image and the target cropping frame intersect, determining the target image based on the second image generated by the adjusted camera includes: cropping the second image according to a second cropping frame to obtain the target image; the center point of the second cropping frame corresponds to the center point of the second image, and the size of the second cropping frame corresponds to the shooting magnification; Alternatively, the second cropping frame is position-transformed according to the jitter offset information to obtain a corrected cropping frame; and the second image is cropped according to the corrected cropping frame to obtain the target image.

6. The method according to any one of claims 1 to 5, wherein: Correcting the first image according to the jitter offset information to obtain a target image includes: When a second preset condition is met, positionally transforming the first cropping frame according to the shake offset information to obtain a target cropping frame; wherein the second preset condition includes that a shooting magnification of the camera is greater than a base magnification, a center point of the first cropping frame corresponds to a center point of the first image, and a size of the first cropping frame corresponds to the shooting magnification; The target image is determined according to the target cropping frame and the first image.

7. The method according to claim 6, wherein The determining the target image according to the target cropping frame and the first image includes: In a case where the first image includes the target cropping frame, cropping the first image according to the target cropping frame to obtain the target image; When the first image does not contain the target cropping frame, or when there is an intersection between the first image and the target cropping frame, align part of the boundary of the target cropping frame with the boundary of the first image to obtain an aligned cropping frame; and crop the first image according to the aligned cropping frame to obtain the target image.

8. The method according to any one of claims 1 to 7, wherein: The acquiring of the jitter offset information of the camera in the terminal device from the moment when the shooting operation is received to the moment when the first image is framed includes: Acquire multiple position offset information of the camera collected by the optical image stabilizer during a time period from the moment the shooting operation is received to the moment the first image is framed; The jitter offset information is determined according to the plurality of position offset information.

9. The method according to any one of claims 1 to 8, wherein The non-zero-second delay mode is any of the following: long exposure mode, night scene mode, the ambient brightness is lower than a preset brightness threshold, the camera does not support zero-second delay, and there is a difference in the processing flow of the preview stream and the photo stream.

10. The method according to any one of claims 3 to 7, wherein: The method further comprises: performing coordinate mapping on four boundaries of the first cropping frame in the first image according to the jitter offset information to obtain positions of four boundaries of the target cropping frame; If all boundaries of the target cropping frame do not exceed boundaries of the first image, determining that the first image includes the target cropping frame; In a case where any boundary of the target cropping frame exceeds a boundary of the first image, it is determined that the first image does not include the target cropping frame, or that there is an intersection between the first image and the target cropping frame.

11. The method according to any one of claims 3 to 7, wherein: The method further comprises: Performing coordinate mapping on four vertices of the first clipping frame in the first image according to the jitter offset information to obtain coordinates of four vertices of the target clipping frame; In a case where all vertices of the target clipping frame are located in an area formed by a boundary of the first image, determining that the first image includes the target clipping frame; If any vertex of the target clipping frame is not located in the area formed by the boundary of the first image, it is determined that the first image does not include the target clipping frame, or that there is an intersection between the first image and the target clipping frame.

12. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 11 when executed by the processor.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 11.

14. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 11.

Images