Display control method, mobile terminal and computer readable storage medium

Abstract

The invention provides a display control method, a mobile terminal and a computer readable storage medium, and relates to the technical field of terminals. The mobile terminal successively receives a first zooming operation, at least one second zooming operation and a third zooming operation, and different zooming operations are different in zooming multiplying power. And the mobile terminal firstly responds to the first zooming operation to obtain a first zooming image according to the zooming magnification of the first zooming operation, then responds to the third zooming operation to correspondingly obtain a third zooming image, and displays the third zooming image. The post-generated third zoom operation is processed and displayed ahead of the previously received at least one second zoom operation. Compared with a zoom display scheme in which all zoom operations are sequentially processed according to the receiving sequence, the time for the mobile terminal to respond to all or part of the image acquisition operation of the at least one second zoom operation is shortened, so that the display delay of the later received zoom operation is shortened, and the followability of the zoom operation is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a display control method, a mobile terminal and a computer-readable storage medium. Background Art

[0002] With the development of science and technology, the functions of mobile terminals such as mobile phones are becoming increasingly powerful. The camera function of mobile terminals can realize front and rear camera, image beautification processing, zoom, etc. Mobile terminals are usually equipped with multiple sets of lenses, and different lenses correspond to different focal lengths. Lenses with different focal lengths can capture images of different distances, different fields of view, and different sharpness. The zoom operation (Zoom) of the mobile terminal is to adjust the positions of several sets of lenses with different focal lengths to achieve the change of the focal length of the adjusted lens to capture images with different focal lengths. When the user uses the mobile terminal to preview the photos, different zoom touch actions can be applied to the photo preview interface of the mobile terminal. The mobile terminal detects the touch parameters of the user's zoom touch action to generate a corresponding zoom request, and sends the generated zoom request to the corresponding functional module to process the zoom request, and displays the final preview image.

[0003] When the user is previewing a photo, there may be different zoom requirements in sequence, so there may be multiple zoom operations triggered, and there will also be multiple corresponding zoom requests. The mobile terminal will process each zoom request in sequence according to the order in which each zoom request is generated and display the corresponding preview image. This will result in the zoom request generated in real time by the mobile terminal being processed some time after being triggered, and the preview image corresponding to the zoom request will also be displayed some time later, and the delay from applying the zoom touch operation to displaying the corresponding zoom preview image is relatively long.

[0004] It can be seen that when the existing mobile terminal performs photo preview, the response delay of the zoom operation is long and the hand tracking performance is poor. Summary of the invention

[0005] The embodiments of the present application provide a display control method, a mobile terminal, and a computer-readable storage medium, which are used to implement a solution of jumping a zoom operation received later to a zoom operation received earlier to queue up and process and display it in advance, thereby reducing the response and display delay of the newly received zoom operation and optimizing the tracking performance of the zoom display.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In the first aspect, a display control method is provided, wherein a mobile terminal receives multiple zoom operations in sequence, including a first zoom operation, at least one second zoom operation, and a third zoom operation in sequence, wherein the first zoom operation is received earlier than the third zoom operation, and there is at least one second zoom operation between the first zoom operation and the third zoom operation, and the zoom magnifications of different zoom operations may be different. The mobile terminal first responds to the first zoom operation, obtains a first zoom image with the zoom magnification of the first zoom operation, and before responding to all second zoom operations or displaying the second zoom images corresponding to all second zoom operations, the mobile terminal first responds to the third zoom operation to obtain a third zoom image, and displays the third zoom image. The process of responding to the zoom processing operation may include in sequence: generating a zoom request, sending a zoom request, image acquisition operation, image signal front-end processing operation, spatial alignment transformation processing operation, image signal back-end processing operation, and image display.

[0008] The display control method provided by the present application adjusts the response display process of the continuous zoom operation, and the third zoom operation generated later will be processed and displayed in advance before the at least one second zoom operation received earlier. Compared with the zoom display scheme in which all zoom operations are processed in sequence according to the order in which they are received, the entire response time or part of the response time of at least one second zoom operation is reduced, so that the display delay of the zoom operation received later is reduced, and the tracking performance of the zoom operation is improved.

[0009] During specific execution, the at least one second zoom operation is not fully responded to, which means that any one of the at least one second zoom operation does not require an image acquisition operation, or it may mean that some of the second zoom operations in the at least one point zoom operation require an image acquisition operation, or only some of the second zoom operations need to be subjected to partial image acquisition operations, such as only performing a spatial alignment transformation processing operation or an image signal back-end processing operation.

[0010] According to a possible implementation of the first aspect, after the first zoom operation is responded to and the corresponding first zoom image is displayed, after receiving the third zoom operation, the zoom image corresponding to the at least one second zoom operation is no longer displayed, but the third zoom operation is directly responded to and the third zoom image is displayed after the first zoom image is displayed. Or only a partial image acquisition operation of a second zoom request after the first zoom image is performed and the corresponding zoom image is not displayed, but directly jumps to the third zoom operation received later. The mobile terminal first displays the first zoom image corresponding to the first zoom operation on the shooting interface, and directly displays the third zoom image corresponding to the third zoom operation. The third zoom image corresponding to the third zoom operation most recently applied by the user will be quickly displayed, and the hand tracking performance is significantly improved.

[0011] In a possible implementation of the first aspect, each time the shooting interface of the mobile terminal receives a zoom operation, it will first generate a corresponding zoom request and add it to the request queue corresponding to the zoom operation. That is, when the mobile terminal receives the first zoom operation, it will generate a corresponding first zoom request and add it to the request queue. Then, the mobile terminal generates a corresponding second zoom request for at least one second zoom operation received and adds it to the request queue. Then, if the mobile terminal receives a third zoom operation, it will also generate a corresponding third zoom request and add it to the request queue. The request queue is a first-in-first-out queue, and the zoom request that joins first will be dequeued first for processing, and the zoom request that joins later will be dequeued later for processing. It should be noted that the mobile terminal may continue to receive multiple zoom operations, so the zoom request entry and exit are also continuous, which does not mean that the newly generated zoom request will be entered into the queue only after the previous zoom request is dequeued.

[0012] The first zoom request that enters the queue first will be dequeued first, and the first zoom request that is dequeued will perform image acquisition operations, including image collection operations, image processing operations, etc., to obtain and display the first zoom image. Similarly, when the third zoom request corresponding to the third zoom operation that enters the queue later is dequeued, the mobile terminal executes the image acquisition operation corresponding to the third zoom request to obtain the third zoom image and display the third zoom image. Different from the existing scheme of first dequeuing at least one second zoom request to perform an image acquisition operation and send it for display before dequeuing the third zoom request to perform an image acquisition operation and send it for display, the present application can advance the processing of the third zoom request before the second zoom request is displayed. Based on the first-in-first-out principle of the request queue, the mobile terminal can replace the second zoom ratio of the second zoom request with the third zoom ratio of the third zoom request, so that the zoom image corresponding to the third zoom ratio is acquired in advance and displayed in advance as the third zoom image. Alternatively, at least one second zoom request before the third zoom request may be dequeued and discarded without performing any image acquisition operation, or only a portion of the image acquisition operation may be performed before performing the image acquisition operation of the third zoom request, thereby ensuring that the third zoom image is acquired and displayed in advance.

[0013] In a possible implementation of the first aspect, when the mobile terminal jumps to respond to the third zoom request, it first selects a second zoom request suitable for jumping from the second zoom requests corresponding to the second zoom operation whose corresponding zoom image has not been displayed before the third zoom operation, as the target zoom request, and the queue entry time of the target zoom request is earlier than the queue entry time of the third zoom request. When the target zoom request is dequeued, the image acquisition operation of the third zoom request is jumped to be executed, and the acquired third zoom image is displayed in advance. There may be multiple bases for selecting the second zoom request suitable for jumping, for example, the selection basis may be which second zoom request has a zoom ratio that is closer to the zoom ratio of the third zoom request, or which second zoom request has an earlier dequeuing time, or which zoom request involves image data or processing procedures that are closer to the image data or processing procedures of the third zoom request.

[0014] After determining a target zoom request to which the third zoom request can be jumped, there may be multiple ways to jump to the third zoom request.

[0015] One implementation method of jumping to respond to the third zoom request is to jump the third zoom ratio of the third zoom request to the target zoom request for early processing. Specifically, the target zoom ratio corresponding to the target zoom request is replaced or modified to the third zoom ratio. In this way, when the target zoom request is dequeued, the target zoom request is actually an image acquisition operation at the third zoom ratio, so the corresponding zoom image is also a third zoom image obtained at the third zoom ratio. In this case, it can be ensured that the target zoom request in the request queue is responded to after being dequeued, and the third zoom image can be obtained and displayed in advance at the third zoom ratio. It should be noted that since the third zoom ratio of the third zoom request has been advanced to the target zoom request for processing, the third zoom request can be directly discarded, or discarded when the third zoom request is dequeued. Part of the second zoom request between the target zoom request and the third zoom request can also be discarded synchronously to ensure that the zoom ratio of the zoom image displayed last is consistent with the zoom ratio of the latest received third zoom operation.

[0016] Another implementation method of jumping to respond to the third zoom request is to process the third zoom request in advance. According to the first-in-first-out principle, when the target zoom request is dequeued, all zoom requests between the target zoom request and the third zoom request can be directly dequeued and discarded, and then the third zoom request is dequeued and the image acquisition operation corresponding to the third zoom request is executed to obtain the third zoom image. In this way, it can also be ensured that the zoom magnification of the zoom image finally displayed is consistent with the zoom magnification of the most recently received zoom operation.

[0017] It should be noted that, in the above two schemes of jumping to respond to the third zoom request, the image acquisition operation of the target zoom request may not be performed at all, or part of the image acquisition operation of the target zoom request may be performed.

[0018] In a possible implementation of the first aspect, considering that the main factor affecting different zoom requests is the zoom ratio, the zoom requests with the same or similar zoom ratios have the same or similar zoom processing operations and the acquired zoom images. Then, when considering the third zoom request to be queued in advance, the third zoom request can be advanced to the second zoom request with a relatively close zoom ratio in the previously generated zoom request as the target zoom request that can be jumped. In specific implementation, the mobile terminal can first set a preset range, which is used to limit the allowable range of the ratio difference between zoom requests suitable for jumping. The preset range can be set according to the focal length of each lens in the mobile terminal. For example, for different zoom ratios, the focal length of the lens corresponding to the zoom ratio can be selected to set different preset ranges. In addition, considering that the queue processing of the zoom request in advance may affect the smoothness and timeliness of the entire zoom display process, the preset ratio can also be set according to the smoothness requirement or timeliness requirement of the mobile terminal for zoom switching. The higher the smoothness requirement, the smaller the preset range is, and the higher the timeliness requirement, the larger the preset range is.

[0019] The mobile terminal first obtains the magnification difference between the second zoom ratio corresponding to each second zoom request and the third zoom ratio of the third zoom request, and selects the second zoom request with the magnification difference within the preset range as the target zoom request to which it can jump. At this time, the selected target zoom request may be one or more. If there is one target zoom request, the target zoom request can be directly used as the zoom request to which the third zoom request can jump. If there are multiple target zoom requests, one can be randomly selected from the multiple target zoom requests as the target zoom request suitable for jumping to the third zoom request. It is also possible to select a target zoom request that is relatively more suitable for jumping from multiple target zoom requests in combination with the real-time processing status of each target zoom request or other zoom parameters such as lens type.

[0020] In a possible implementation of the first aspect, a solution is provided for selecting a target zoom request suitable for jumping according to the real-time status of each zoom request. After receiving the zoom operation acting on the shooting interface, the mobile terminal will immediately generate a corresponding zoom request, and add the zoom request to the request queue to wait for a response. According to the first-in-first-out principle, the state of the request in the request queue is the waiting state for dequeuing, and the real-time state of the request dequeued from the request queue changes from the waiting state for dequeuing to the dequeued state.

[0021] Furthermore, there may be multiple processing states for zoom requests that have been dequeued, including: request issuance, waiting for image acquisition, image signal front-end processing, spatial alignment transformation processing, image signal back-end processing, display, etc. Among them, the original image acquisition operation can be performed by the camera sensor in the mobile terminal, the zoom request is issued to the camera driver, the camera driver controls the camera sensor to acquire the original image, and transmits the acquired original image to the subsequent data processing module. The data processing module mainly includes three modules, which are: an image signal front-end processing module for preliminary cropping of the original image, a spatial alignment transformation processing module for calculating the cropping data, and an image signal back-end processing module for re-cropping based on the cropping data. After the image signal back-end processing module performs re-cropping, a zoom image corresponding to the zoom request can be obtained, and the zoom image is sent to the camera application of the mobile terminal for display, and the camera application displays the received zoom image on the shooting interface.

[0022] Relatively speaking, the processing progress of the second zoom request in the dequeued state is faster than that in the waiting state. If the third zoom request is queued to the second zoom request in the dequeued state and processed in advance, the zoom delay is shorter and the hand tracking is better. The image acquisition operation in the waiting state has not started. If the third zoom request is queued to the second zoom request in the waiting state, all image acquisition operations corresponding to the third zoom request will be executed when the second zoom request is waiting to be dequeued, which can also achieve the effect of processing the third zoom request in advance and displaying the image.

[0023] In a possible implementation of the first aspect, the mobile terminal can combine the zoom ratio and the real-time status to comprehensively select a target zoom request suitable for the third zoom request to jump. Specifically, the mobile terminal selects the second zoom request whose zoom ratio difference with the third zoom ratio is within a preset range and whose real-time status is that it has been dequeued as the target zoom request. In this way, the smoothness and timeliness of the entire zoom display effect are guaranteed after the third zoom request is processed in advance.

[0024] In a possible implementation of the first aspect, considering that in the actual image acquisition operation process, there may be two or more zoom requests in the mobile terminal that are in the dequeued state and have not yet been displayed, these zoom requests may be in the data acquisition node, the image signal front-end processing node, the space alignment transformation processing node, the image signal back-end processing node, the waiting display node, etc., and these zoom requests can be used as target zoom requests to which the third zoom request can jump. Then, after screening out the candidate zoom requests whose zoom magnifications and the magnification difference between the third zoom magnification are within the preset range and whose real-time status is the dequeued state, the zoom requests with relatively later nodes of the image acquisition operation among these candidate zoom requests can be used as target zoom requests, for example, the candidate zoom requests whose image acquisition operation is in the image signal back-end processing operation or the space transformation processing operation are given priority as target zoom requests to which the third zoom request can jump. In this way, if the selected target zoom request is already in the image signal back-end processing operation or the spatial alignment transformation processing operation, the target zoom ratio of the target zoom request can be modified to the third zoom ratio of the third zoom request, and the remaining image acquisition operations, such as image signal processing operations and spatial alignment transformation processing operations, can be performed at the third zoom ratio. The third preview image can be obtained relatively quickly at the third zoom ratio, further reducing the response and display delay of the third zoom request.

[0025] In a possible implementation of the first aspect, the influence of target zoom requests in different real-time states on the image acquisition operation of the third zoom request is described in detail. The above-mentioned step of jumping to execute the image acquisition operation corresponding to the third zoom request to obtain the third zoom image and displaying the third zoom image when the target zoom request is out of the queue may include: on the one hand, the real-time state of the target zoom request is a waiting state for dequeueing, that is, there are one or more second zoom requests waiting to be processed or being processed before the target zoom request. Then, according to the first-in-first-out principle, it is necessary to wait for the target zoom request to be out of the queue first. Only when the target zoom request is out of the queue, jump to execute all image acquisition operations corresponding to the third zoom request to obtain the third zoom image. In this case, the second zoom request and the target zoom request before the third zoom request can be directly discarded, or the third zoom ratio of the third zoom request can be used as the target zoom ratio of the target zoom request to perform all image acquisition operations in advance and obtain the corresponding third zoom image.

[0026] On the other hand, if the real-time status of the target zoom request is the dequeued status, according to the processing progress of the target zoom request, the image acquisition operation of the target zoom request may only have a spatial alignment transformation processing operation or an image signal backend processing operation left. At this time, the target zoom ratio corresponding to the target zoom request can be replaced with the third zoom ratio, and after the zoom ratio is replaced, the remaining image acquisition operations of the target zoom request are performed with the replaced third zoom ratio to quickly obtain the third zoom image.

[0027] In a possible implementation of the first aspect, the relevant software modules for implementing the above-mentioned display control in the mobile terminal are explained. Specifically, the mobile terminal includes: a camera application, a camera service and a hardware abstraction layer. The mobile terminal usually includes an application layer, a framework layer, a hardware abstraction layer and a kernel layer from top to bottom, wherein the application layer runs a camera application and other functional applications, can receive user input operations, and display relevant image data, etc. The framework layer is equipped with a camera service, etc., and provides an application programming interface and a programming framework for the upper-layer camera application and other functional applications. The kernel layer is a layer between hardware and software. The kernel layer at least includes a camera driver. The hardware abstraction layer can encapsulate the driver in the kernel layer, and provide a calling interface to the framework layer, shielding the implementation details of the underlying hardware.

[0028] Each time the camera application receives a zoom operation acting on the shooting interface, the camera application sends a zoom parameter corresponding to the zoom operation to the camera service; the camera service generates a zoom request corresponding to the zoom operation and adds it to the request queue, and sends each zoom request in the request queue to the hardware abstraction layer in sequence; the hardware abstraction layer executes an image acquisition operation corresponding to each zoom request to obtain a zoom image, and sends the obtained zoom image to the camera application; the camera application displays the zoom image.

[0029] Specifically, the hardware abstraction layer includes an interface module, a camera sensor and a data processing module; the interface module sends each received zoom request to the camera sensor; the camera sensor performs exposure and image output operations according to each zoom request to obtain a corresponding original image; the data processing module performs data processing operations on the original image of each zoom request to obtain a corresponding zoom image, and sends the zoom image to the camera application.

[0030] In a possible implementation of the first aspect, in order to implement the display control method provided by the present application, the mobile terminal further includes a selection module, and the hardware abstraction layer further includes an application module. The selection module determines the target zoom request to which the third zoom request can jump; the application module modifies the target zoom ratio of the target zoom request to the third zoom ratio; the data processing module executes the image acquisition operation corresponding to the modified target zoom request to obtain the third zoom image. A specific functional module is added to realize the queue-jumping selection and implementation scheme of the third zoom request.

[0031] In a possible implementation of the first aspect, a scheme is also provided for selecting a target zoom request to be jumped to according to the real-time status of each previous second zoom request. After each second zoom request is added to the queue, the real-time status is divided into two types, waiting for dequeue status or dequeue status. The zoom request in the dequeue status has been issued and is in the node for performing a specific image acquisition operation, which includes an initial image acquisition operation node, an image signal front-end processing operation node, and an image signal back-end processing operation node in sequence according to the timing of the image acquisition operation.

[0032] Relatively speaking, the zoom request in the dequeued state will be responded to earlier than the zoom request in the waiting dequeued state. Furthermore, among these zoom requests in the dequeued state, the zoom requests in the spatial alignment transformation processing operation node and the signal backend processing operation node will be responded to earlier. Then, in order to display the third zoom image corresponding to the third zoom request in advance, the zoom request whose real-time state is the dequeued state can be selected. If there are multiple zoom requests in the dequeued state, the second zoom request whose real-time operation node after dequeuing is the image signal backend processing operation node or the spatial alignment transformation processing operation node can be selected and determined as the target zoom request. This solution of only referring to the real-time state without relying on the zoom ratio to select the jumpable target zoom request is more suitable for one-way zoom scenarios in which the zoom ratio is gradually increased or gradually decreased, for example, the one-way zoom scenario in which the zoom ratio is gradually increased from 1x to 10x, or the zoom ratio is gradually reduced from 20x to 1x.

[0033] In a second aspect, a mobile terminal is provided, which has the function of implementing the display control method described in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0034] In a third aspect, a mobile terminal is provided, comprising: a camera application, a processor and a memory, wherein the camera module and the memory are coupled to the processor; the memory is used to store computer execution instructions, and when the mobile terminal is running, the processor executes the computer execution instructions stored in the memory, so that the mobile terminal executes the display control method as described in any one of the above-mentioned first aspects.

[0035] In a fourth aspect, a display control device is provided, comprising: a receiving module, a response module and a display module. The receiving module is used to receive multiple zoom operations, and sequentially receives a first zoom operation, at least one second zoom operation and a third zoom operation, the first zoom operation is received earlier than the third zoom operation, and the first zoom operation is separated from the third zoom operation by at least one second zoom operation. The response module is used to respond to a first zoom operation among the multiple zoom operations, and the display module is used to display a first zoom image, wherein the first zoom image is an image obtained at a first zoom ratio corresponding to the first zoom operation; the response module is also used to respond to a third zoom operation among the multiple zoom operations, and display a third zoom image, wherein the third zoom image is an image obtained at a third zoom ratio corresponding to the third zoom operation.

[0036] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer can execute the display control method described in any one of the above-mentioned first aspects.

[0037] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the display control method described in any one of the first aspects.

[0038] Among them, the technical effects brought about by any design method in the third to sixth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of a shooting interface of a mobile terminal;

[0040] Figure 2 This is one of the schematic diagrams for comparing interfaces before and after a mobile terminal receives a zoom operation in an existing solution;

[0041] Figure 3 For the corresponding Figure 2 Schematic diagram of the expected optimization effect;

[0042] Figure 4 This is the second schematic diagram of the interface comparison before and after the mobile terminal receives the zoom operation in the existing solution;

[0043] Figure 5 For the corresponding Figure 4 Schematic diagram of the expected optimization effect;

[0044] Figure 6 A schematic diagram of the internal software architecture of a mobile terminal provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the internal flow of a mobile terminal processing a zoom request provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the internal flow of a mobile terminal processing multiple zoom requests provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of a flow chart of implementing zoom request jump processing by a display control method provided in an embodiment of the present application;

[0048] Fig.10 A schematic diagram showing a comparison of the processes before and after the mobile terminal application display control method provided in an embodiment of the present application;

[0049] Fig.11 A flow chart of a display control method provided in an embodiment of the present application;

[0050] Fig.12 A schematic diagram of implementing a request queue interruption using the display control method provided in an embodiment of the present application;

[0051] Fig.13 A schematic diagram of the structure of a mobile terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0053] To facilitate understanding, some technical common sense involved in the embodiments of the present application is first introduced.

[0054] The mobile terminal is provided with lenses of different focal lengths. By adjusting the zoom parameters such as the zoom ratio (Zoom Ratio) of the zoom to adjust the field of view (FOV) of the shooting interface, a technical effect of enlarging or reducing the photographed object in the shooting interface is achieved. In the shooting, video recording or photo preview scene, in order to obtain images of different sizes of the photographed object, the user applies a zoom operation on the shooting interface of the mobile terminal. The camera application and related modules in the mobile terminal cooperate to respond to the zoom operation to obtain a zoomed image corresponding to the zoom ratio and display it on the shooting interface. The main application scenarios of the display control method provided in the embodiment of the present application include shooting preview scenarios of mobile phones, and the shooting preview scenarios of mobile phones will be described below. However, the provided display control method is not limited to being applied only to mobile phones, and is not limited to being applied only to shooting preview scenarios.

[0055] like Figure 1 The figure shows a schematic diagram of a shooting interface of a mobile terminal. In the shooting interface, the user can perform a zoom operation, and the mobile terminal receives the zoom operation and responds to the zoom operation to obtain a corresponding zoom image and display the zoom image in the shooting interface. Figure 1 As shown, the shooting interface 100 of the mobile terminal mainly includes: a parameter control area 101, an image display area 102, a mode control area 103, a front and rear camera flip control 104, a gallery control 105, a shutter control 106 and a zoom control 107. Among them, the parameter control area 101 includes a plurality of parameter controls, each of which is used to respond to the shooting parameter adjustment operation input by the user. The parameter controls included in the parameter control area may include, but are not limited to: a flash control, an AI recognition switch control, a color standard control, and a more detailed camera setting control. The image display area 102 can be used to display a preview image, which is an image collected by the mobile terminal in real time through a camera. The mobile terminal can refresh the display content in the image display area 102 in real time to facilitate the user to preview the image currently collected by the camera. The mode control area 103 may include a plurality of mode controls corresponding to different shooting modes, such as an aperture mode control, a night scene mode control, a portrait mode control, a photo mode control, a video mode control, a professional mode control, and more mode controls. Each mode control can be marked only by text information, for example, "aperture", "night scene", "portrait", "photo", "video", "professional", "more", or it can be displayed by an icon, or a combination of text information and icons.

[0056] The zoom control 107 is used to respond to the zoom operation of the user on the mobile terminal to adjust the FOV of the image display area 102. The "1 device" displayed on the zoom control 107 indicates that the current optical zoom ratio of the mobile terminal is 1x, and the subsequent x represents the zoom ratio. The user can apply the zoom operation by applying a point operation or a sliding operation on the zoom control 107, and of course, the zoom operation can also be applied by a touch operation that does not act on the zoom control 107. For example, the mobile terminal can pre-define the operation of sliding fingers toward each other or sliding fingers away from each other on the shooting interface as a zoom operation, or pre-define the operations of clockwise rotation sliding and counterclockwise rotation sliding on the shooting interface as zoom operations. Such other pre-defined touch operations act on the shooting interface, but not necessarily on the zoom control 107.

[0057] Of course, in other cases, the zoom operation received by the mobile terminal may also be a zoom operation that does not need to act on the shooting interface, but an adjustment operation on a physical device on the surface or side of the mobile terminal. For example, a physical button or knob is set on the side of the mobile terminal as a zoom switch, and this part of the physical buttons or knobs can be associated with indicating a specific zoom value selection or zoom ratio adjustment operation, etc. This part of the physical buttons or knobs can be a separate physical device dedicated to receiving zoom operations, or it can be a reused existing volume adjustment button or channel adjustment knob in a shooting scene, etc., without limitation. This situation may be more suitable for scenarios where the mobile terminal is a retro-styled mobile phone, an elderly phone, a card machine, etc.

[0058] After the mobile terminal receives the zoom operation applied by the user, it can respond to the zoom operation. The process of responding to the zoom operation may include but is not limited to: generating a zoom request according to the zoom operation, sending the zoom request to perform image acquisition operation and image processing operation, obtaining a zoom image, and finally sending the zoom image to the shooting interface for display. In actual operation, the user may apply multiple zoom operations in succession on the shooting interface, such as applying multiple zoom operations by sliding the zoom control, or applying multiple zoom operations by continuously clicking the zoom control. Then, the mobile terminal will also receive these multiple zoom operations in succession, and respectively generate a zoom request for each received zoom operation, perform image acquisition operation and image processing operation after the zoom request is sent, obtain the zoom image and send it for display. This is a complete set of zoom operation response processes. Under normal circumstances, each zoom operation response process requires a processing time of approximately 250 milliseconds. Then, if the user releases the hand after applying multiple zoom operations in succession, the mobile terminal will execute the zoom operation response process of each zoom operation in sequence according to the sequence of the zoom operations. The last zoom operation needs to wait for all the previous zoom operation response processes to be completed before it can be responded to, and the zoom image corresponding to the last zoom operation needs to be displayed after the last zoom operation response process is completed. As a result, the waiting time from receiving the last zoom operation to displaying the zoom image corresponding to the last zoom operation is long, that is, the delay of the zoom operation is long and the hand tracking performance is poor.

[0059] like Figure 2 As shown, it is a comparative schematic diagram showing that in a scenario where a zoom operation is applied in a sliding touch mode, there is a delay in the existing zoom preview interface after the zoom operation is applied. Figure 2 As shown in (a), the user's finger operates the sliding control 107 at 1x, the zoom ratio 108 on the shooting interface shows 1x, and the corresponding zoom image is a zoom image of 1x. Figure 2 As shown in (b) of FIG. 1 , the user slides the sliding control 107 from 1x to 3x. At this time, the zoom ratio 108 on the shooting interface has also been switched from 1x to 3x. However, due to the response delay, the zoom image in the image display area 102 does not switch to the 3x zoom image, but still displays the 1x zoom image, which has poor chirality. The shooting interface we expect is as follows Figure 3 As shown in (a) to (b), the user's finger slides the sliding control 107 from 1x to 3x. At this time, the zoom ratio 108 on the shooting interface has also been switched from 1x to 3x, and the zoom image in the image display area 102 has also been switched to a 3x zoom image. The user can clearly feel the improvement of the zoom tracking performance.

[0060] like Figure 4As shown in FIG. 1 , a comparison diagram of a scene in which a finger slides relative to a touch control to apply a zoom operation, and a conventional zoom preview interface having a delay after applying a zoom operation is shown. Figure 4 As shown in (a) to (b) in the figure, the user's fingers slide back and forth to increase the zoom ratio. At this time, the sliding control 107 automatically switches from 1x to 3x. At this time, the zoom ratio 108 on the shooting interface has also switched from 1x to 3x. However, due to the response delay, the zoom image in the image display area 102 does not switch to the 3x zoom image, but still displays the 1x zoom image, which has poor chirality. Figure 5 As shown in (a) to (b) in the figure, the shooting interface expected by our mobile phone is that the user slides his fingers away from each other to increase the zoom ratio. At this time, the sliding control 107 automatically switches from 1x to 3x. At this time, the zoom ratio 108 on the shooting interface has also switched from 1x to 3x, and the zoom image in the image display area 102 switches from a 1x zoom image to a 3x zoom image, with good tracking performance.

[0061] In order to obtain better hand-tracking performance, the embodiment of the present application provides a display control method and a mobile terminal and a computer-readable storage medium for implementing the display control method, and the zoom request generated later is jumped to a certain previous zoom request for processing in advance, so as to reduce the response delay of the zoom request generated later. The provided display control method is applied to a mobile terminal equipped with a camera and related support modules. The mobile terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a special camera (such as a SLR camera, a card camera), etc., without limitation.

[0062] The mobile terminal receives the zoom operation performed by the user on the shooting interface, and generates a zoom preview image corresponding to the zoom touch operation by calling the camera application and related modules running in the internal software architecture of the mobile terminal. It should be noted that the camera application applicable to the embodiment of the present application may include any application on the mobile terminal that can realize the shooting preview function, such as the original camera application, the image beautification camera application, etc., without limitation.

[0063] like Figure 6 The figure shows the internal architecture of the mobile terminal. The process of the mobile terminal responding to the zoom operation will be explained in detail below in combination with the internal architecture of the mobile terminal.

[0064] Specifically, the internal architecture of the mobile terminal can be divided into four layers, from top to bottom, namely, the application layer (APP), the framework layer (FWK), the hardware abstraction layer (HAL), and the kernel layer (or driver layer). It should be noted that in addition to these main functional layers, other functional modules may also be included without limitation.

[0065] The application layer can include a series of application packages, such as camera applications, gallery applications, applications with camera functions, etc. Application packages can also include applications such as calls, calendars, maps, navigation, music, videos, short messages, etc.

[0066] The framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0067] The framework layer runs a camera service, which can be called by the camera application to implement functions related to shooting. In addition, the framework layer can also include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for the mobile terminal. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, notification information is used to inform the completion of downloads, message reminders, etc. Notification information can also be a notification that appears in the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. Notification information can also be, for example, a text message prompted in the status bar, a prompt sound, a vibration of the power terminal, a flashing indicator light, etc. It should be pointed out that the camera application can also call the content provider, resource manager, notification manager, window manager, view system, etc. according to actual business needs, and the embodiments of the present application do not impose any restrictions on this.

[0068] The kernel layer is a layer between hardware and software. The kernel layer includes at least a camera driver. The camera driver can be used to drive a hardware module with a shooting function, such as a camera sensor. In other words, the camera driver is responsible for data interaction with the camera sensor. The kernel layer may also include a display driver, an audio driver, a sensor driver, etc., and the embodiments of the present application do not impose any restrictions on this.

[0069] The hardware abstraction layer can encapsulate the drivers in the kernel layer and provide a calling interface to the framework layer, shielding the implementation details of the underlying hardware. Figure 6As shown, the hardware abstraction layer may include a camera call processing module (Camera HAL), a multi-camera decision module, etc. The camera call processing module is the core software framework of the camera, and the camera call processing module includes an interface module, a sensor node (Sensor Node) and a data processing module, etc. In one implementation, the data processing module may include an image signal front-end processing (Image Singnal Processing Front End, IFE) module, a spatial alignment transform (Spatial Alignment Transform, SAT) processing module, and an image signal back-end processing (Image Singnal Processing Post End, IPE) module. Among them, the related processing involved in the sensor node and the image signal front-end processing module belongs to the processing of the real-time pipeline (Realtime Pipeline), and the related processing involved in the control alignment transform processing module and the image signal back-end processing module belongs to the processing of the offline pipeline (Offline Pipeline). The above-mentioned sensor node, data processing module and interface module are components in the image data and control instruction transmission pipeline in the camera call processing module.

[0070] Specifically, the sensor node can be a control node for the camera sensor, and the sensor node can control the camera sensor through the camera driver. The interface module can be a software interface for the application framework layer, which is used to interact with the application framework layer for data. Of course, the interface module can also interact with the multi-camera decision module, data processing module, sensor node, etc. in the camera call processing module. The data processing module can process the original image data sent back by the camera sensor, wherein the image signal front-end processing module is used to process the preview image collected by the camera sensor for preliminary cropping and reserve the image edge (Margin); the spatial alignment transformation processing module is used to perform spatial alignment on the image data according to the zoom ratio and the spatial alignment transformation processing algorithm, and determine the cropping data and the warp data to make the preview image smoother; the image signal back-end processing module is used to perform cropping and warping on the preview image data according to the calculation results of the spatial alignment transformation processing module.

[0071] The camera application can pass the camera mode, zoom parameters and other information selected by the user to the camera service of the framework layer, and then the camera service passes it to the multi-camera decision module through the interface module of the hardware abstraction layer. The multi-camera decision module can determine the camera sensor to be used for image output according to the application scenario, such as the camera sensor of the front camera or the camera sensor of the rear camera, and the camera sensor of the main camera, the camera sensor of the wide-angle camera, and the camera sensor of the telephoto camera in the rear camera.

[0072] In a mobile terminal with a zoom function, its multi-camera decision module can also determine the target camera to be switched according to the user's zoom operation, and turn on the camera sensor of the target camera in advance through the camera driver. In addition, the multi-camera decision module can also set the camera that currently outputs the image among multiple cameras with turned on camera sensors.

[0073] Understandably, Figure 6 The layers in the software structure shown and the components contained in each layer do not constitute a specific limitation on the mobile terminal. In other embodiments of the present application, the mobile terminal may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, which is not limited in the present application.

[0074] In addition, it is understandable that in order to implement the zoom response method in the embodiment of the present application, the mobile terminal includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0075] In the above Figure 6 Based on the internal architecture diagram shown, Figure 7 The camera application and related supporting modules in the mobile terminal are retained, and the process of the mobile terminal responding to the zoom operation is illustrated.

[0076] The process of a mobile terminal responding to a zoom operation can be divided into two parts: top-down signal flow ( Figure 7 The mobile terminal receives the zoom operation and generates a corresponding control signal to control the relevant modules to perform the image acquisition operation to obtain the zoom image. The data flow from bottom to top ( Figure 7 As shown by the dashed arrow in the middle, the camera sensor of the mobile terminal acquires the original image, obtains the zoomed image after image processing, and sends the zoomed image to the camera application of the application layer for display.

[0077] Based on the above Figure 6In the internal architecture layer of the mobile terminal shown, in the preview scenario of the camera application, the process of each functional module cooperating to perform the zoom touch operation can mainly include: receiving a zoom touch operation, generating a zoom request, processing the zoom request to obtain a preview image, and sending the preview image for display. In actual use, the mobile terminal may continue to receive multiple zoom touch operations and generate multiple zoom requests. The mobile terminal processes each zoom request in turn to generate a corresponding preview image and sends it for display. In order to reduce the response delay of the latest zoom touch operation, the display control method provided in the embodiment of the present application processes the latest zoom touch operation in advance, changing the normal order in which the mobile terminal processes multiple continuous zoom touch operations. For ease of understanding, the conventional process of a mobile terminal processing a zoom touch operation is explained below.

[0078] like Figure 7 The signal flow shown by the solid arrow in the middle mainly includes: the user applies a zoom operation by clicking, sliding or other pre-defined operation methods on the shooting preview interface of the mobile terminal, and the touch module of the mobile terminal responds to the zoom touch operation and generates a corresponding touch point reporting event and sends it to the camera application of the application layer. After receiving the touch point reporting event, the camera application converts the point reporting coordinate data corresponding to the touch point reporting event into a zoom ratio (Zoom Ratio) and sends it to the camera service of the framework layer. The camera service generates a zoom request corresponding to the zoom ratio, and sends the zoom request to the camera hardware call module in the hardware abstraction layer. The zoom request is sent to the sensor node. According to the zoom request, the sensor node controls the camera sensor through the camera driver to perform exposure and image output operations.

[0079] like Figure 7 The data flow shown by the dotted arrow in the figure mainly includes: after the camera sensor completes the exposure and image output operation, the original image data corresponding to the zoom request is sent to the image signal front-end processing module through the camera driver. The image signal front-end processing module performs preliminary cropping processing on the original image data corresponding to the zoom request, and sends the processed image data to the spatial alignment transformation processing module. Among them, the image signal front-end processing module can perform preliminary cropping processing on the original image data according to the zoom ratio corresponding to the zoom request, and retain the edge area of ​​the image, so that the spatial alignment transformation processing module can perform spatial alignment processing on the image. The spatial alignment transformation processing module performs image spatial alignment processing according to the zoom ratio corresponding to the zoom request and the relevant spatial alignment transformation processing algorithm, calculates the cropping data and distortion data corresponding to the image data, and sends the calculation results and the image data corresponding to the zoom request to the image signal back-end processing module. The image signal back-end processing module performs another cropping operation and image distortion operation on the image data corresponding to the zoom request according to the calculation results of the spatial alignment transformation processing, obtains a preview image corresponding to the zoom request, and then sends the preview image to the camera application of the application layer for preview display.

[0080] It should be noted that in the process of the original image output from the camera sensor passing through the image signal front-end processing module, the spatial alignment transformation processing module, the image signal back-end processing module and then being sent to the display, in addition to the data stream of image data transmission, a signal stream containing various data processing instructions and parameters is also transmitted.

[0081] Above Figure 7 The figure shows the situation where a mobile terminal processes a zoom request. In the preview scenario of the camera application, the mobile terminal may continue to receive multiple zoom operations. Then, the internal camera application and related supporting modules of the mobile terminal need to respond to these multiple zoom operations in sequence. After each zoom operation is received, the camera application will generate a corresponding zoom ratio based on the touch point event corresponding to the zoom operation, and send the zoom ratio to the camera service, which will generate a corresponding zoom request. Then, in the case of continuously receiving multiple zoom operations, the camera service will continue to generate multiple corresponding zoom requests, and the zoom ratios of different requests may be different.

[0082] like Figure 8 As shown, the camera service can maintain a request queue and add each newly generated zoom request to the request queue. The request queue is a first-in-first-out queue. The first zoom request that enters the queue first is sent to the hardware abstraction layer for image acquisition operations. The zoom request that has not been processed waits for processing in the request queue. The newly generated zoom request is added to the end of the request queue. Figure 8 As shown, assuming that 30 zoom requests are currently being generated continuously, the newly generated zoom request 30 is at the end of the request queue, all zoom requests before zoom request 20 have been processed, zoom request 20 to zoom request 24 have been dequeued and all image acquisition operations have not yet been completed, and zoom request 25 to zoom request 30 are all in the request queue waiting to be dequeued and responded to.

[0083] Depend on Figure 8 It can be seen that each zoom request issued by the camera service undergoes data processing operations such as sensor node collecting image data, preliminary cropping, calculating cropping data and distortion data, re-cropping and image distortion in the camera hardware call module before the corresponding preview image can be obtained and displayed. The time from reception to preview display for each zoom operation is long and the response is slow.

[0084] When actually using the camera preview function, the user will continue to apply multiple zoom operations by clicking, sliding, etc. to indicate different zoom requirements, but usually the latest generated zoom request is closer to the user's actual zoom requirement. Some of the previous zoom touch operations before the latest zoom request may be zoom touch operations that are accidentally triggered during the sliding process, or zoom requirements that the user needed at the previous moment but may no longer need at the current moment. That is, after the zoom request is generated, some of the previous zoom requests before the zoom request may no longer need to be processed and previewed, which is a factor that allows the zoom request to be queued up in advance.

[0085] On the other hand, the image data sources and data processing operations involved in processing different zoom requests are related to the zoom parameters such as the zoom ratio and the focal length of the lens corresponding to each zoom request to a certain extent. Figure 8 From the corresponding zoom request processing flow, it can be seen that after the zoom request is sent to the camera hardware call module, the sensor node first controls the camera sensor through the camera driver to perform exposure and image output operations. The controlled camera sensor is the camera sensor corresponding to the zoom ratio. In the subsequent data processing, the original preview image collected by the camera sensor is relied on to perform the initial cropping process corresponding to the zoom ratio, calculate the cropping and distortion data, and perform re-cropping and distortion processing. In the preview scene, the camera shooting object usually does not change. If the zoom ratios of different zoom requests are close (for example, the difference is within 0.5x, 1x or 1.5x), or they belong to the focal length range of the same lens type, then the original preview image obtained by the exposure and image output operation at the sensor node is also relatively close or even the same, and the subsequent initial cropping process, calculation of cropping and distortion data, or the data or calculation amount of re-cropping and distortion processing will also be relatively close. That is, when different zoom requests are processed, there may be similar or identical image acquisition or data processing operations due to relatively close or identical zoom parameters. In this case, in order to save processing time, the original preview image obtained by a previous zoom request, the intermediate data obtained after preliminary cropping, re-cropping and distortion, etc. may also be considered. This is another factor that allows the third zoom request to be processed in the queue.

[0086] In one example, the zoom magnifications of the previously generated zoom requests 20-29 in the request queue may be "2.8x, 2.0x, 3.5x, 2.8x, 2.5x, 1x, 2.5x, 3.5x, 3x, 3.2x," respectively, and the zoom magnification of the latest generated zoom request 30 may be "3.1x." Then, when the user applies the zoom operation corresponding to the latest generated zoom request 30, the mobile terminal may be processing the zoom request 20 in advance, or may be processing the previous zoom requests 20-22, that is, multiple different previous zoom requests may all be in the processing state, but the processing nodes of the multiple previous zoom requests being processed are different. Then, some previous zoom requests between the latest zoom request 30 and the previous zoom request 20 being processed or the previous zoom requests 20-22, such as the previous zoom requests 23-29, may no longer be the user's actual zoom requirements. In this case, it is possible to consider advancing the latest generated zoom request to the time when this part of the previous zoom requests is processed or to queue up after they are processed.

[0087] like Fig. 9 As shown, the display control method provided in an embodiment of the present application is a schematic diagram of advancing the subsequent zoom operation to the previously generated zoom request for queue processing at the internal software architecture level.

[0088] At the application level, the mobile terminal will continue to receive multiple zoom operations, and define the zoom operation with the earliest reception time among the multiple zoom operations that are not currently displayed as the first zoom operation, and the zoom operation with the latest reception time as the third zoom operation, and at least one zoom operation received between the first zoom operation and the third zoom operation is the second zoom operation. In terms of display order, after the mobile terminal displays the first zoom image, it will directly display the third zoom operation, and will not display the second zoom image corresponding to the second zoom operation. Then, at least one of the second zoom operations in between may not need to be responded to at all, or only some of the second zoom operations may need to be responded to. Fig. 9 As shown, zoom request 20 has completed the image acquisition operation and is about to be displayed, corresponding to the first zoom image of the first zoom operation. Zoom request 30 is the latest generated third zoom request, and the zoom requests between zoom request 21 to zoom request 29 correspond to the second zoom request. Before processing the second zoom request and displaying the corresponding second zoom image (such as the zoom image of zoom request 21), the third zoom request (zoom image of zoom request 30) is jumped to display.

[0089] like Fig.10 As shown, it is a schematic diagram of the internal image acquisition operation and external image display of the display control method.

[0090] Fig.10(a) is a schematic diagram of the case where the third zoom request is not responded to in advance, that is, the mobile terminal will respond to the first zoom request, at least one second zoom request and the third zoom request in sequence, and display the first zoom image, at least one second zoom image and the third zoom image in sequence.

[0091] Fig.10 (b) is a schematic diagram of the case where the third zoom request is responded to in advance, that is, the mobile terminal will respond to the third zoom request after responding to the first zoom request, and will interrupt the display of the third zoom image after displaying the first zoom image.

[0092] contrast Fig.10 As can be seen from (a) and (b), the third zoom operation applied by the user later will be responded to and displayed quickly, which can intuitively improve the user's zoom experience.

[0093] In addition, considering that the zoom magnifications of different zoom requests may vary greatly, if a previous zoom request is randomly selected to jump in line, the switching effect of the preview image may be poor due to the large difference in zoom magnifications. The display control method provided in the embodiment of the present application considers first selecting a zoom request suitable for the third zoom request to jump from multiple previous second zoom requests, and jumping the third zoom request to the selected second zoom request for processing. For example, the aforementioned Fig.10 In the example, the zoom ratios of zoom request 23 (zoom ratio 2.8x), zoom request 28 (zoom ratio 3x), etc. in at least one of the second zoom requests are relatively close to the zoom ratio (zoom ratio 3.1x) of the third zoom request (i.e., zoom request 30), so it is possible to consider advancing the third zoom request to the time when these zoom requests with relatively close zoom ratios are processed or to queue up after they are processed. In this way, the response delay of the subsequent zoom requests can be reduced, and the stability and smoothness of the zoom preview display can be optimized.

[0094] like Fig.11 FIG. 1 is a flow chart of selecting a target zoom request in a display control method provided in an embodiment of the present application. The flow chart mainly includes the following steps:

[0095] Step S1101, selecting a jumpable target zoom request from the second zoom request;

[0096] Step S1102, when the target zoom request is dequeued, jump to execute the graphics acquisition operation of the third zoom request;

[0097] Step S1103: displaying the third zoom image of the third zoom request in advance.

[0098] The request queue maintained by the mobile terminal may include: a first zoom request (zoom request 20) to be displayed, multiple previously generated second zoom requests (zoom requests 21-29) whose corresponding zoom images have not yet been displayed, and a newly generated third zoom request (zoom request 30). The generation time of these zoom requests is arranged in sequence along the direction of the arrow, that is, the generation time of the prior zoom request 20 is the earliest, the generation time of the prior zoom request 21 is only later than the prior zoom request 20, and the generation time of the third zoom request 30 is the latest.

[0099] like Fig.11 As shown, from the second zoom requests 21-29, a second zoom request K is selected as the target zoom request, K∈ any integer in (20-29). When the mobile terminal selects a target zoom request that can jump to the queue for the third zoom request, it can consider the impact of the zoom ratio on the amount of calculation or the impact on the smoothness of the display switching, and select the target zoom request according to the zoom ratio of each second zoom request. Of course, it is also possible to select a target zoom request suitable for queue-jumping processing based on the real-time status of each second zoom request, whether it is waiting to be dequeued in the request queue, or the status of data processing operation after being dequeued. In other cases, the mobile terminal can also select a target zoom request with a lens matching from multiple prior second zoom requests according to the lens type required by the third zoom request, etc., without limitation.

[0100] After determining the target zoom request to which the third zoom request can jump according to the aforementioned steps, the third zoom request can be processed in advance. There can be multiple specific advance schemes. For example, after the data processing operation of the target zoom request is completed, the data processing operation of the third zoom request can be jumped to execute. Alternatively, the data processing operation of the third zoom request can also be jumped to execute when the target zoom request completes a certain processing node, such as completing the image signal front-end processing node, or completing the spatial alignment transformation processing node, or completing the image signal back-end processing node.

[0101] When processing the target zoom request, the mobile terminal processes the third zoom request in advance, and after processing, the third zoom image corresponding to the third zoom request can be obtained. Then, at this time, the mobile terminal can preview and display the third zoom image. The third zoom image is previewed and displayed, and its timely response zoom effect can be directly seen by the user, which is also the basis for the user to judge the zoom experience. The latest generated zoom request is processed and displayed in advance, which greatly optimizes the user's photo preview experience.

[0102] See above for further details. Fig. 9The mobile terminal also includes a selection module, and the hardware abstraction layer also includes an application module; wherein the selection module determines a target zoom request to which the third zoom request can jump; the application module modifies the target zoom ratio of the target zoom request to the third zoom ratio; and the data processing module executes an image acquisition operation corresponding to the modified target zoom request to obtain the third zoom image.

[0103] In addition, in specific implementation, it may be considered to first complete the data processing operation of the target zoom request to obtain the corresponding target preview image, and then display the latest preview image corresponding to the third zoom request after displaying the target preview image, so as to optimize the smoothness of the zoom image switching. Of course, in order to further reduce the response delay, it may also be considered not to complete the data processing operation of the target zoom request, or not to display the target preview image corresponding to the target zoom request, but to directly display the third zoom image after displaying the zoom image of the previous zoom request of the target zoom request, so as to improve the timeliness of the zoom preview display.

[0104] It should be noted that the display control method provided in the embodiment of the present application is more suitable for the situation where there are three or more zoom requests. In this case, the delay of the zoom response is relatively long, which will obviously affect the user experience. If there is only one zoom request, that is, there are no other second zoom requests that have not been processed before the third zoom request, then there is no need to determine the target zoom request, and the third zoom request can be directly processed after the first zoom request is processed. The response delay here only includes the processing time of the third zoom request, and does not include the processing time waiting for other prior zoom requests. If there is only one prior second zoom request before the latest third zoom request, then it is also possible to directly wait until the prior second zoom request is processed before processing the third zoom request, or directly use the second zoom request as the target zoom request. Of course, the third zoom request can also be directly jumped to the processing before the second zoom request, etc., without limitation.

[0105] The display control method provided by the embodiment of the present application can be configured as a self-starting scheme for the zoom preview scene, that is, the mobile terminal will apply the display control method provided by the embodiment of the present application in real time or periodically when it is in the zoom preview scene. For example, each time a new zoom request is generated, the newly generated zoom request can be used as the third zoom request, and a target zoom request that can be jumped to the queue is selected for it. If a new zoom request (such as zoom request 31) is generated after the newly generated zoom request (such as zoom request 30), then the zoom request generated earlier (such as zoom request 30) becomes the previous zoom request of the zoom request (such as zoom request 31) generated later, and the zoom request (such as zoom request 30) generated earlier can also be queued by the zoom request (such as zoom request 31) generated later during the process of waiting for the advance queue processing, so as to give priority to the timely response effect of the latest generated zoom request. This configuration scheme may be more suitable for zoom preview scenarios where users expect the zoom preview image to be displayed in time.

[0106] Of course, the display control method provided in the embodiment of the present application can also be configured as a solution manually enabled by the user. That is, the user can apply a corresponding touch operation in the setting interface of the mobile terminal or in the shooting control of the shooting preview interface to indicate the application of the display control method provided in the embodiment of the present application. This configuration scheme may be more suitable for zoom preview scenarios where the user expects the zoom preview image to display different zoom effects in sequence.

[0107] By the aforementioned Figures 9 to 11 The process of determining to respond to the third zoom request in advance based on the zoom ratio is shown below. Then, how to select a target zoom request suitable for the third zoom request jump based on the zoom ratio will be explained in detail below.

[0108] Basically, the mobile terminal can select a second request whose zoom magnification is close to the zoom magnification of the third zoom request as the target zoom request. For example, the magnification difference between the second zoom magnification and the third zoom magnification corresponding to each of the second zoom requests is obtained, and the second zoom request whose magnification difference is within a preset range is selected as the target zoom request.

[0109] In other cases, the mobile terminal may also select a zoom magnification that is smaller than or larger than the zoom magnification of the third zoom request, or select a previous zoom request whose zoom magnification meets other preset conditions as the target zoom request, etc.

[0110] After each second zoom request is added to the queue, the real-time status is divided into two types, waiting to be dequeued or already dequeued. The zoom request in the already dequeued state has been issued and is at a node that performs a specific image acquisition operation, which includes an initial image acquisition operation node, an image signal front-end processing operation node, and an image signal back-end processing operation node in sequence according to the timing of the image acquisition operation. The real-time status of the second zoom request is different, and its remaining waiting time is different. For example, the previous zoom request in the waiting to be dequeued state has a remaining waiting time that includes at least its own entire processing time, and may also include the processing time for waiting for other prior zoom requests. In the second zoom request that is already dequeued, its remaining waiting time may include part of its own processing time. The remaining waiting time of the second zoom request that is already dequeued is relatively short, and all processing operations may be completed earlier.

[0111] On the one hand, the mobile terminal may select a scheme of a jumpable target zoom request only according to the real-time status of each previous second zoom request.

[0112] Relatively speaking, the zoom request in the dequeued state will be responded to earlier than the zoom request in the waiting dequeued state. Furthermore, among these zoom requests in the dequeued state, the zoom requests in the spatial alignment transformation processing operation node and the signal backend processing operation node will be responded to earlier. Then, in order to display the third zoom image corresponding to the third zoom request in advance, the zoom request whose real-time state is the dequeued state can be selected. If there are multiple zoom requests in the dequeued state, the second zoom request whose real-time operation node after dequeuing is the image signal backend processing operation node or the spatial alignment transformation processing operation node can be selected and determined as the target zoom request. This solution of only referring to the real-time state without relying on the zoom ratio to select the jumpable target zoom request is more suitable for one-way zoom scenarios in which the zoom ratio is gradually increased or gradually decreased, for example, the one-way zoom scenario in which the zoom ratio is gradually increased from 1x to 10x, or the zoom ratio is gradually reduced from 20x to 1x. The solution of selecting the target zoom request only by referring to the real-time status can improve the timeliness of the third zoom request, but may affect the smoothness of preview image switching, etc.

[0113] On the other hand, the mobile terminal may also combine the zoom ratio and the real-time status to select a target zoom request that is more suitable for the third zoom request to jump into the queue. The specific step of determining the target zoom request may include:

[0114] The mobile terminal first determines the zoom magnification of the third zoom request based on the zoom magnification of the third zoom request and the dependency of similar zoom magnifications in the data processing operation, and obtains a magnification range. Then, based on the magnification range, a part of the second zoom requests that may be suitable are screened out from all the second zoom requests, and then a target zoom request that is suitable for the third zoom request to jump to is further selected from the part of the second zoom requests. For ease of description, the part of the previous zoom requests can be defined as candidate zoom requests.

[0115] The mobile terminal first determines the magnification to which the zoom magnification of the third zoom request belongs. Specifically, the mobile terminal predefines a magnification step according to the correlation between the zoom magnifications of each focal length, and the magnification step indicates the range of zoom magnifications with relatively large correlation. The mobile terminal determines the magnification range to which the zoom magnification belongs with the zoom magnification of the third zoom request as the center, wherein the length of the magnification range is equal to the preset adjustment step, and the median of the magnification range is the zoom magnification of the third zoom request. For example, the preset adjustment step predefined by the mobile terminal is 1x. If the zoom magnification of the third zoom request is 3.1x, then the corresponding magnification range may be (2.6x-3.6x). If the preset adjustment step is 0.4x, then the corresponding magnification range may be (2.9x-3.3x).

[0116] Of course, there may be other ways to determine the magnification range, for example, the magnification range of the preset adjustment step before the zoom magnification of the third zoom request may be taken. The preset adjustment step in the example is 0.4x, and the zoom magnification of the third zoom request is 3.1x, then the corresponding magnification range may be (2.7x-3.1x), etc. Of course, there may be other ways to select the magnification range, which are not limited.

[0117] In the process of selecting a target zoom request from candidate zoom requests, if there are at least two candidate zoom requests, the candidate zoom request whose image acquisition operation is in the image signal backend processing operation or the space alignment transformation processing operation is used as the target zoom request.

[0118] All processing operations of each zoom request in the data processing module include image signal front-end processing operation, spatial alignment transformation processing operation and image signal back-end processing operation in sequence. The processing progress of the spatial alignment transformation processing operation and the image signal back-end processing operation is relatively fast. When both the zoom ratio and the real-time state are satisfied, the candidate zoom request whose image acquisition operation is in the image signal back-end processing operation or the spatial alignment transformation processing operation can be preferentially selected as the target zoom request.

[0119] It should be noted that, in the aforementioned implementation, the candidate zoom requests whose zoom magnifications belong to the magnification range and whose real-time status is that they have been dequeued are selected from the prior zoom requests. The candidate zoom requests mentioned here are only used to refer to some zoom requests that meet the requirements, and are not limited to being assigned or renamed as candidate zoom requests. That is, the mobile terminal can only perform the action of screening according to the zoom magnification and magnification range, and does not need to perform the action of assigning or renaming objects for the part of the prior zoom requests obtained by the screening action.

[0120] In other cases, the mobile terminal can also superimpose the zoom lens type to select the target zoom request. The lenses installed in the mobile terminal mainly include the main camera lens, the wide-angle lens and the telephoto lens. The main camera lens is the one with the highest pixel among all the cameras of the mobile terminal. The wide-angle camera is a camera with a wide-angle function. The wide-angle viewing angle is wider than that of a general lens and the focal length is shorter. It is often used to shoot objects with a large area. The telephoto camera can take longer photos to ensure the clarity of the photos. Wide-angle lenses and telephoto lenses can usually make up for the shortcomings of the main camera and improve the shooting effect. The zoom parameters of the zoom request generated by the mobile terminal may also include the zoom lens type, and the zoom lens type includes at least one of the main camera lens, the telephoto lens and the wide-angle lens.

[0121] The mobile terminal determines the target zoom request according to the zoom lens type of the third zoom request and the zoom lens types of each candidate zoom request. Specifically, the candidate zoom request whose zoom lens type includes the zoom lens type of the third zoom request is used as the target zoom request.

[0122] For example, as shown in the above example, the zoom magnification of zoom request 20 is 2.8x, and its zoom lens type may be a telephoto lens. The zoom magnification of zoom request 21 is 2.0x, and its zoom lens type may be a main camera lens. The zoom magnification of the third zoom request 30 is 3.1x, and its zoom lens type may be a telephoto lens. Then, the zoom switching type from zoom request 20 to the third zoom request 30 is Zoom In, and the zoom lens type (main camera lens) included in zoom request 20 does not include the zoom lens type (telephoto lens) included in the third zoom request 30. The zoom switching type from zoom request 21 to the third zoom request 30 is Zoom Out, and the zoom lens type (telephoto lens) included in zoom request 21 includes the zoom lens type (telephoto lens) included in the third zoom request 30. The scheme of selecting zoom request 21 as the target zoom request reduces the operation of switching lenses to regenerate the original image and align it compared to the scheme of selecting zoom request 20 as the target zoom request.

[0123] After determining the target zoom request based on the above implementation modes, the mobile terminal may have multiple options for the node to which the third zoom request is redirected in advance.

[0124] In one implementation, if the real-time status of the target zoom request is waiting to be dequeued, the third zoom request may be advanced to after the target zoom request, and when the target zoom request is dequeued, the image acquisition operation of the third zoom request may be executed.

[0125] In another implementation, if the real-time state of the target zoom request is in the dequeued state, the real-time state of the third zoom request may be jumped to the current processing node of the target zoom request. For example, if the real-time node of the target zoom request is a spatial alignment transformation processing node, the real-time node of the third zoom request is jumped from the waiting node to the spatial alignment transformation processing node.

[0126] Of course, considering the continuity between processing nodes, that is, some processing nodes may have forward-associated processing nodes on the processing link, or backward-associated processing nodes, each zoom request needs to undergo processing operations of each processing node in turn on the processing link before obtaining the final preview image. For example, if the second zoom request 21 is in the spatial alignment transformation processing node, the earlier first focus request 20 may be in the image signal back-end processing node, the later second zoom request 22 may be in the image signal front-end processing node, and the later previous zoom request 23 may be in the sensor node. Then, when jumping into the queue, it is also possible to consider jumping the real-time node of the third zoom request from the waiting node to a processing node before the spatial alignment transformation processing node, that is, the image signal front-end processing node, and wait for the target zoom request to enter its backward-associated image signal back-end processing node in the spatial alignment transformation processing node, and the third zoom request enters the spatial alignment transformation processing node from the image signal front-end processing node.

[0127] During specific execution, if the real-time status of the target zoom request is a dequeued status, the target zoom ratio corresponding to the target zoom request is modified to the third zoom ratio, and the remaining image acquisition operations of the modified target zoom request are executed to obtain the third zoom image. Based on the first-in-first-out principle of the request queue, the mobile terminal can replace the second zoom ratio of the second zoom request with the third zoom ratio of the third zoom request, so that the zoom image corresponding to the third zoom ratio is acquired in advance and displayed in advance as the third zoom image. Alternatively, at least one second zoom request before the third zoom request can be dequeued and then discarded without executing any image acquisition operation, or only a portion of the image acquisition operation is executed before executing the image acquisition operation of the third zoom request, which can also ensure that the third zoom image is acquired and displayed in advance.

[0128] In addition, there are other display control schemes in the embodiments of the present application. Fig.12 As shown in FIG. 1 , if there are zoom requests 1 to 7 in the request queue that has been generated previously and has not yet been displayed, and zoom requests 8 to 14 are received in sequence thereafter, the zoom magnifications of the zoom requests are as follows: Fig.12 Then, when the display control is performed based on the zoom ratio, the process of selecting the jump scheme for each subsequent zoom request 8 to 14 is as follows:

[0129] The zoom ratio 1.6x of zoom request 8 is close to the zoom ratio 1.5x of zoom request 3, zoom request 8 jumps to zoom request 3, and the zoom ratio of zoom request 3 is changed to 1.6x of zoom request 8;

[0130] The zoom ratio 1.9x of zoom request 9 is close to the zoom ratio 2.0x of zoom request 4, so zoom request 9 first jumps to zoom request 4, and the zoom ratio of zoom request 4 is modified to the zoom ratio 1.9x of zoom request 9;

[0131] The zoom ratio 2.1 of zoom request 10 is close to the zoom ratio 1.9x of zoom request 9, and zoom request 10 then jumps to zoom request 9 (actual zoom request 4), and the zoom ratio of zoom request 4 is modified to the zoom ratio 2.1x of zoom request 10;

[0132] The zoom ratio 2.6x of zoom request 11 is close to the zoom ratio 2.5x of zoom request 5, zoom request 11 jumps to zoom request 5, and the zoom ratio of zoom request 5 is modified to the zoom ratio 2.6x of zoom request 11;

[0133] The zoom ratio 3.0 of zoom request 12 is close to the zoom ratio 3.0 of zoom request 7 (same is also considered close), zoom request 12 jumps to zoom request 7, and the zoom ratio of zoom request 7 is modified to the zoom ratio 3.0x of zoom request 12;

[0134] Zoom request 13 has no previous zoom request suitable for queue interruption, so zoom request 13 can be reserved in order and placed after zoom request 7 after the zoom magnification is modified, forming a new zoom request 8 with a zoom magnification of 3.5x;

[0135] The zoom ratio of zoom request 14 is close to that of zoom request 13 (new zoom request 8 ), so zoom request 14 is redirected to new zoom request 8 , and the zoom ratio of new zoom request 8 is modified to the zoom ratio of zoom request 14 (3.6x).

[0136] In this way, we can get Fig.12 The new zoom request processing sequence and the actual zoom ratio are shown in the sequence on the right. Fig.12It can be clearly concluded that the subsequent zoom request is processed in advance, which reduces the response delay caused by the image processing operation waiting for multiple zoom requests.

[0137] In summary, the display control method provided in the embodiment of the present application combines the zoom parameters such as the zoom ratio of the latest generated third zoom request and the zoom ratios of each previously generated zoom request, selects a target zoom request suitable for the third zoom request to jump into the queue from multiple previously generated zoom requests, and jumps the third zoom request to the target zoom request for processing and displays it in advance, so as to reduce the response delay of the third zoom request and optimize the tracking performance of the zoom preview.

[0138] In addition, an embodiment of the present application further provides a mobile terminal, including a camera module, a memory and a processor, wherein the camera module is coupled to the memory and the processor;

[0139] Memory stores computer-executable instructions;

[0140] The processor executes the computer-executable instructions stored in the memory, so that the mobile terminal executes the display control method provided by the above embodiment. In addition to these main components, the mobile terminal also includes components for realizing basic functions. Fig.13 Provide specific instructions.

[0141] like Fig.13 The figure shows a schematic diagram of the structure of a mobile terminal 1300 provided in an embodiment of the present application. The mobile terminal 1300 may include a processor 1310, a memory 1320, a camera 1330, a display screen 1340, a touch sensing module 1350, a motor 1360, an audio module 1370, a sensor module 1380, a button 1390, etc. The sensor module 1380 may include a gyroscope sensor 1380A, an acceleration sensor 1380B, a distance sensor 1380C, a proximity light sensor 1380D, an ambient light sensor 1380E, etc.

[0142] The structure illustrated in the embodiment of the present invention does not constitute a limitation on the mobile terminal 1300. It may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0143] The processor 1310 may include one or more processing units. For example, the processor 1310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0144] The controller can be a decision maker that directs the various components of the mobile terminal 1300 to work in coordination according to the instructions. It is the nerve center and command center of the mobile terminal 1300. The controller generates an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0145] The processor 1310 may also be provided with a memory 1320 for storing instructions and data. In some embodiments, the memory 1320 in the processor 1310 is a high-speed cache memory, which can store instructions or data that the processor 1310 has just used or cyclically used. If the processor 1310 needs to use the instruction or data again, it can be directly called from the memory 1320. This avoids repeated access, reduces the waiting time of the processor 1310, and thus improves the efficiency of the system.

[0146] In some embodiments, the processor 1310 may include an interface. The interface may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0147] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDL) and a serial clock line (SCL). In some embodiments, the processor 1310 may include multiple groups of I2C buses. The processor 1310 may be coupled to the touch sensor, charger, flash, camera 1330, etc. through different I2C bus interfaces. For example: the processor 1310 may be coupled to the touch sensor through the I2C interface, so that the processor 1310 communicates with the touch sensor through the I2C bus interface to realize the touch function of the mobile terminal 1300.

[0148] The I2S interface can be used for audio communication. In some embodiments, the processor 1310 can include multiple groups of I2S buses. The processor 1310 can be coupled to the audio module 1370 via the I2S bus to achieve communication between the processor 1310 and the audio module 1370. In some embodiments, the audio module 1370 can transmit an audio signal to the communication module via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0149] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 1370 and the communication module can be coupled via a PCM bus interface. In some embodiments, the audio module 1370 can also transmit audio signals to the communication module via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication, and the sampling rates of the two interfaces are different.

[0150] The UART interface is a universal serial data bus for asynchronous communication. The bus is a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 1310 and the communication module. For example, the processor 1310 communicates with the Bluetooth module via the UART interface to implement the Bluetooth function. In some embodiments, the audio module 1370 can transmit an audio signal to the communication module via the UART interface to implement the function of playing music via a Bluetooth headset.

[0151] The MIPI interface can be used to connect the processor 1310 with peripheral devices such as the display screen 1340 and the camera 1330. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 1310 and the camera 1330 communicate via the CSI interface to implement the shooting function of the mobile terminal 1300. The processor 1310 and the display screen 1340 communicate via the DSI interface to implement the display function of the mobile terminal 1300.

[0152] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 1310 with the camera 1330, the display 1340, the communication module, the audio module 1370, the sensor module 1380, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0153] The USB interface may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface may be used to connect a charger to charge the mobile terminal 1300, or may be used to transmit data between the mobile terminal 1300 and a peripheral device. The USB interface may also be used to connect headphones to play audio through the headphones. The USB interface may also be used to connect other mobile terminals, such as AR devices, etc.

[0154] The interface connection relationship between the modules shown in the embodiment of the present invention is only for illustrative purposes and does not constitute a structural limitation on the mobile terminal 1300. The mobile terminal 1300 may adopt different interface connection modes in the embodiment of the present invention, or a combination of multiple interface connection modes.

[0155] The wireless communication function of the mobile terminal 1300 can be implemented by an antenna, a radio frequency module, a communication module, a modem, and a baseband processor. The antenna is used to transmit and receive electromagnetic wave signals. Each antenna in the mobile terminal 1300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna.

[0156] The mobile terminal 1300 implements the display function through a GPU, a display screen 1340, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 1340 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1310 may include one or more GPUs that execute program instructions to generate or change display information.

[0157] The display screen 1340 is used to display images, videos, etc. The display screen 1340 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLED, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile terminal 1300 may include 1 or N display screens 1340, where N is a positive integer greater than 1.

[0158] The mobile terminal 1300 can implement a shooting function through an ISP, a camera 1330, a video codec, a GPU, a display screen, and an application processor.

[0159] The ISP is used to process the data fed back by the camera 1330. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and chromaticity of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 1330.

[0160] The camera 1330 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the mobile terminal 1300 may include 1 or N cameras 1330, where N is a positive integer greater than 1.

[0161] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the mobile terminal 1300 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0162] The video codec is used to compress or decompress digital video. The mobile terminal 1300 may support one or more video codecs. Thus, the mobile terminal 1300 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG13, MPEG4, etc.

[0163] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, the intelligent cognition of the mobile terminal 1300 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0164] The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 1300. The external memory card communicates with the processor 1310 through the external memory interface to implement a data storage function, such as storing music, video and other files in the external memory card.

[0165] The internal memory can be used to store computer executable program codes, which include instructions. The processor 1310 executes various functional applications and data processing of the mobile terminal 1300 by running the instructions stored in the internal memory. The memory 1320 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile terminal 1300 (such as audio data, a phone book, etc.), etc. In addition, the memory 1320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, other volatile solid-state storage devices, a universal flash memory (Universal Flash Storage, UFS), etc.

[0166] The mobile terminal 1300 can implement audio functions, such as voice-controlled photo taking, music playing and recording, etc., through the speaker, receiver, microphone, headphone jack, and application processor of the audio module 1370 .

[0167] The audio module 1370 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 1370 can also be used to encode and decode audio signals. In some embodiments, the audio module 1370 can be arranged in the processor 1310, or some functional modules of the audio module 1370 can be arranged in the processor 1310.

[0168] The speaker, also called a "horn", is used to convert an audio electrical signal into a sound signal. The mobile terminal 1300 can listen to music or listen to a hands-free call through the speaker.

[0169] The receiver, also called a "handset", is used to convert audio electrical signals into sound signals. When the mobile terminal 1300 receives a call or voice message, the voice can be received by placing the receiver close to the human ear.

[0170] A microphone, also called a "microphone" or "microphone", is used to convert sound signals into audio electrical signals. When making a call or sending a voice message, the user can put his mouth close to the microphone and speak to input the sound signal into the microphone. The mobile terminal 1300 can be provided with at least one microphone. In some embodiments, the mobile terminal 1300 can be provided with two microphones, which can not only collect sound signals but also realize noise reduction function. In some embodiments, the mobile terminal 1300 can also be provided with three, four or more microphones to realize the collection of sound signals, noise reduction, identification of sound sources, and realization of directional recording function, etc.

[0171] The headphone jack is used to connect wired headphones. The headphone jack can be a USB interface, or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0172] The key 1390 includes a power key, a volume key, etc. The key 1390 may be a mechanical key or a touch key. The mobile terminal 1300 receives the key 1390 input and generates a key signal input related to the user settings and function control of the mobile terminal 1300.

[0173] Motor 1360 can generate vibration prompts. Motor 1360 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Touch operations acting on different areas of the display screen 1340 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization. In addition, the motor can also be used in the camera zoom shooting scene to achieve lens switching of different focal lengths by driving the movement of the lens.

[0174] The sensor module 1380 may include a gyro sensor 1380A, an acceleration sensor 1380B, a distance sensor 1380C, a proximity light sensor 1380D, and an ambient light sensor 1380E. The gyro sensor 1380A and the acceleration sensor 1380B may be used for posture correction or horizontal positioning during the photographing process, the distance sensor 1380C may be used for depth of field determination or image processing during the photographing process, and the proximity light sensor 1380D and the ambient light sensor 1380E may be used for sensing ambient light and adjusting image brightness values ​​during the photographing process.

[0175] The display control methods in the aforementioned embodiments can all be implemented in the mobile terminal 1300 having the aforementioned hardware structure.

[0176] On the basis of the above embodiments, an embodiment of the present application further provides a display control device, the control device comprising a processor, and the processor is used to execute the display control method provided by the above embodiments.

[0177] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer is enabled to execute the display control method provided in the above embodiment.

[0178] An embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the display control method provided in the above embodiment.

[0179] The specific implementation methods of the mobile terminal, display control device, computer-readable storage medium, and computer program product containing instructions provided in the embodiments of the present application and the technical effects brought about by them can be found in the specific implementation process of the display control method provided in the aforementioned embodiments and the technical effects brought about by them, which will not be repeated here.

[0180] In some embodiments, through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0181] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0183] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A display control method, characterized in that: include: In a process of receiving a plurality of zoom operations, in response to a first zoom operation among the plurality of zoom operations, displaying a first zoom image, wherein the plurality of zoom operations include the first zoom operation, at least one second zoom operation, and a third zoom operation received in sequence, and the first zoom image is an image obtained at a first zoom ratio corresponding to the first zoom operation; Before displaying all the second zoom images, a third zoom image is displayed, wherein one of the second zoom images is an image obtained at a zoom factor corresponding to the second zoom operation, and the third zoom image is an image obtained at a third zoom factor corresponding to the third zoom operation.

2. The display control method according to claim 1, characterized in that: In the process of receiving the plurality of zoom operations, before the step of displaying the first zoom image in response to the first zoom operation among the plurality of zoom operations, the method further includes: Whenever a zoom operation is received, a zoom request corresponding to the zoom operation is generated and added to a request queue; The step of displaying a first zoom image in response to a first zoom operation among the plurality of zoom operations comprises: When the first zoom request corresponding to the first zoom operation is dequeued, acquiring the first zoom image corresponding to the first zoom request, and displaying the first zoom image; The step of displaying the third zoom image comprises: When the third zoom request corresponding to the third zoom operation is dequeued, the third zoom image corresponding to the third zoom request is acquired, and the third zoom image is displayed.

3. The display control method according to claim 2, characterized in that: The step of acquiring the third zoom image corresponding to the third zoom request and displaying the third zoom image comprises: selecting a target zoom request to which the third zoom request can jump, wherein the target zoom request is a second zoom request corresponding to a second zoom operation for which a corresponding zoom image has not yet been displayed; When the target zoom request is dequeued, the third zoom image corresponding to the third zoom request is obtained, and the third zoom image is displayed.

4. The display control method according to claim 3, characterized in that: The step of obtaining the third zoom image corresponding to the third zoom request when the target zoom request is dequeued includes: When the target zoom request is dequeued, an image acquisition operation corresponding to the target zoom request is performed at the third zoom magnification to obtain the third zoom image.

5. The display control method according to claim 3, characterized in that: The step of obtaining the third zoom image corresponding to the third zoom request when the target zoom request is dequeued includes: When the target zoom request is dequeued, the third zoom request and the previous zoom request are dequeued, and the third zoom image corresponding to the third zoom request is directly acquired.

6. The display control method according to any one of claims 3 to 5, characterized in that: The step of selecting a target zoom request to which the third zoom request can jump includes: Acquire a magnification difference between the second zoom magnification corresponding to each of the second zoom requests and the third zoom magnification; The second zoom request with a magnification difference within a preset range is selected as the target zoom request.

7. The display control method according to any one of claims 3 to 6, characterized in that: The step of selecting a target zoom request to which the third zoom request can jump includes: Acquire a real-time status corresponding to each of the second zoom requests, wherein the real-time status includes a waiting to be dequeued status and a dequeued status; The target zoom request is determined based on the real-time status of each of the second zoom requests.

8. The display control method according to claim 7, characterized in that: The step of determining the target zoom request based on the real-time status of each of the second zoom requests comprises: A second zoom request whose zoom magnification difference with the third zoom magnification is within a preset range and whose real-time status is a dequeued status is selected as the target zoom request.

9. The display control method according to any one of claims 4 to 8, characterized in that: All image acquisition operations corresponding to each zoom request include: original image acquisition operation, image signal front-end processing operation, space alignment transformation processing operation and image signal back-end processing operation.

10. The display control method according to claim 9, characterized in that: The step of selecting a zoom magnification and a third zoom magnification, wherein the magnification difference between the selected zoom magnification and the third zoom magnification is within a preset range and the real-time state is a second zoom request in an out-of-queue state as the target zoom request comprises: Screening out candidate zoom requests, wherein the candidate zoom request is a second zoom request whose zoom magnification difference with the third zoom magnification is within a preset range and whose real-time status is a dequeued status; If there are at least two candidate zoom requests, the candidate zoom request whose image acquisition operation is in the image signal backend processing operation or the space alignment transformation processing operation is used as the target zoom request.

11. The display control method according to claim 9 or 10, characterized in that: The step of obtaining the third zoom image corresponding to the third zoom request when the target zoom request is dequeued includes: If the real-time status of the target zoom request is a waiting state for dequeuing, when the target zoom request is dequeued, jump to execute all image acquisition operations corresponding to the third zoom request to obtain the third zoom image; If the real-time status of the target zoom request is a dequeued status, the remaining image acquisition operations of the target zoom request are performed with the third zoom magnification to obtain the third zoom image.

12. The display control method according to claim 7, characterized in that: The step of determining the target zoom request based on the real-time status of each of the second zoom requests comprises: The second zoom request whose real-time state is the dequeued state and whose real-time operation node after being dequeued is an image signal backend processing operation node or a space alignment transformation processing operation node is determined as the target zoom request.

13. A mobile terminal, characterized in that: It includes a camera module, a memory and a processor, wherein the camera module, the memory and the processor are coupled; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the mobile terminal executes the display control method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the display control method according to any one of claims 1 to 12.

Images