Image processing method, terminal and storage medium

Abstract

The invention discloses an image processing method, a terminal and a storage medium, and relates to the technical field of terminals, the terminal can display a first interface, the first interface comprises a floating window of a first application, and the floating window of the first application comprises an image collected by a first camera in real time; in response to opening a second application, opening a second camera, and in the process of opening the second camera, displaying an image collected by the first camera in real time on a floating window of the first application; carrying out flow distribution on the second application; a second interface is displayed after flow distribution is conducted on the second application, the second interface comprises a floating window of the first application and an image collected by the second camera in real time, and in the second interface, the floating window of the first application comprises the image collected by the first camera in real time. The method enables the first application and the second application to access the camera in parallel.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method, a terminal, and a storage medium. Background Technology

[0002] Currently, many devices are equipped with multiple cameras; for example, mobile phones typically include a front-facing camera and a rear-facing camera. With the rapid development of terminal technology, users' demands for device usage are constantly increasing. In more and more scenarios, users expect different applications on the device to access different cameras in parallel. For instance, a third-party application on the device can use the front-facing camera for video calls; during the video call, the user expects the device to respond to the system camera's activation, allowing access to the rear-facing camera.

[0003] However, in practice, when different applications access different physical cameras in parallel, some of these applications often cannot access the corresponding cameras properly. For example, user A uses a third-party application for a video call, which uses the front-facing camera to display user A's image in the video call interface. During the video call, if user A also opens the system camera to take a picture, the system camera will use the rear-facing camera to capture the image. In this case, the third-party application may be unable to access the front-facing camera properly, resulting in user A's image not being displayed in the video call interface or user A's image freezing on the video call screen. Summary of the Invention

[0004] This application provides an image processing method, a terminal, and a storage medium, enabling different applications to access different physical cameras in parallel.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, this application provides an image processing method applied to a terminal. The terminal displays a first interface, which includes a floating window of a first application. This floating window includes an image captured in real-time by a first camera. The terminal can open a second application and a second camera in response to the first interface. Opening the camera is equivalent to Opencamera; for example, the terminal can call the Open() function to open the second camera. During the process of opening the second camera, the image captured in real-time by the first camera is displayed in the floating window of the first application. That is, during the Opencamera process, the first application can still access the first camera. After the Opencamera stage corresponding to the second application, a streaming stage corresponding to the second application is entered, which is the Configure_stream stage. In the Configure_stream stage corresponding to the second application, the terminal performs streaming for the second application. After streaming the second application, the terminal displays a second interface, which includes the floating window of the first application and the image captured in real-time by the second camera. In the second interface, the floating window of the first application includes the image captured in real-time by the first camera.

[0007] In the above solution, the images captured in real time by the first and second cameras can be displayed on the second interface, allowing the first and second applications to access the cameras normally in parallel. That is, it can realize the scenario where multiple applications can access or successfully access different cameras in parallel.

[0008] In one possible implementation of the first aspect, during the Configure_stream phase of the second application, the terminal can obtain the Sensor Settings actually selected for the second camera, i.e., the first driver setting parameters. Then, during the stream allocation process for the second application, the terminal can determine whether processor resources supporting the operation of the second camera exist based on the first driver setting parameters selected for the second camera. If processor resources supporting the operation of the second camera exist, the terminal can issue the first driver setting parameters to the second camera; the first driver setting parameters are used to control the second camera to acquire images.

[0009] Since the actual selected first driver settings parameters can more accurately reflect the resource usage information of the second camera during operation, in the above scheme, the determination of whether there are processor resources to support camera operation based on the actual selected Sensor Settings during the Configure_stream stage (i.e., the stream allocation stage) can obtain a more accurate judgment result. Therefore, it can prevent situations where resources are sufficient from being misjudged or reduce the situation where resources are misjudged as insufficient. This reduces the situation where the camera access sessions of some applications are closed or disconnected due to misjudgment when multiple applications concurrently access different physical cameras, enabling multiple applications to successfully access different physical cameras in parallel in more scenarios.

[0010] Furthermore, the aforementioned solution determines the availability of processor resources to support camera operation based on the selected Sensor Settings, and then handles the situation accordingly, such as allocating resources specifically, rather than assuming the camera needs to occupy the same amount of processor resources in all usage scenarios. This achieves dynamic resource adjustment and allocation to a certain extent, enabling more rational resource utilization. Therefore, it reduces situations where sufficient resources exist, but unreasonable resource allocation prevents the application from successfully accessing the camera. For example, it reduces instances where the application is blocked from accessing the camera due to insufficient resources but unreasonable allocation or misjudgment, thus avoiding frequent camera blocking and reducing application crashes, thereby improving application stability to some extent.

[0011] In one possible implementation of the first aspect, the terminal can exit the second application and display the first interface. In response to opening the third application, the terminal can open the second camera. During the opening of the second camera, the real-time image captured by the first camera is displayed in the floating window of the first application. That is, during the Opencamera phase corresponding to the third application, the first application can still access the camera. After the Opencamera phase corresponding to the third application, the streaming phase corresponding to the third application begins, and streaming is performed on the third application. During the streaming of the third application, the real-time image captured by the first camera is no longer displayed in the floating window of the first application. For example, the display of the image captured by the first camera in the floating window of the first application can be stopped (i.e., no longer any image captured by the first camera is displayed), or, during the streaming of the third application, only a still image is displayed in the floating window of the first application.

[0012] After allocating the bandwidth to the third application, a third interface is displayed. The third interface includes a floating window of the first application and images captured in real time by the second camera. In the third interface, the floating window of the first application does not include images captured by the first camera, or the floating window of the first application displays a still image.

[0013] In the above scheme, during the Opencamera phase corresponding to the third application, the first application can still access the first camera. However, during the streaming process for the third application, specifically during the Configure_stream phase, the real-time image captured by the first camera is no longer displayed. In other words, during the Configure_stream phase for the third application, the first application is in a state of being evicted by the camera and can no longer access the first camera. This means that the first application's inability to access the first camera is determined by the resource usage during the Configure_stream phase, thus accurately evicting the camera and reducing the likelihood of misjudgments preventing the first and third applications from accessing different cameras in parallel.

[0014] In one possible implementation of the first aspect, during the distribution of traffic to the third application, the terminal can obtain the second driver setting parameters actually selected for the second camera. These second driver setting parameters are automatically selected to meet the image acquisition requirements of the third application using the second camera; that is, the images acquired by the second camera using the second driver setting parameters can fully meet the image acquisition requirements of the third application. The terminal can determine whether there are processor resources currently supporting the operation of the second camera based on the second driver setting parameters selected for the second camera. If there are no processor resources currently supporting the operation of the second camera, the terminal stops the first application from accessing the first camera, thus preventing the display of images acquired by the first camera in the first application's floating window during the distribution of traffic to the third application, or displaying a still image in the first application's floating window during the distribution of traffic to the third application.

[0015] In the above scheme, since the actually selected second driver settings parameters can more accurately reflect the resource usage information of the second camera during operation, the determination of whether there are processor resources supporting camera operation during the Configure_stream stage (i.e., the stream allocation stage) can obtain a more accurate result. Only when the processor resources supporting the second camera are more accurately determined will the first application's access to the first camera be stopped. Therefore, this reduces false positives and, to some extent, reduces the possibility of some applications' camera access sessions being closed or disconnected due to false positives when multiple applications concurrently access different physical cameras.

[0016] In one possible implementation of the first aspect, the terminal can exit the second application and display the first interface.

[0017] In response to opening the fourth application, the terminal can open the second camera. During the opening of the second camera, the real-time image captured by the first camera is displayed in a floating window of the first application. That is, during the Opencamera phase corresponding to the fourth application, the first application can still access the camera. After the Opencamera phase corresponding to the fourth application, the flow allocation phase corresponding to the fourth application begins, where the fourth application is allocated its flow. During the flow allocation process for the fourth application, the terminal can adjust the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameters; the first setting item in the fourth driver setting parameters corresponds to the second parameter value; the second parameter value is less than the first parameter value. By lowering the first parameter value corresponding to the first setting item in the third driver setting parameters, the processor resource consumption of the second camera can be reduced. Therefore, the terminal can issue the fourth driver setting parameters to the second camera; the fourth driver setting parameters are used to control the image acquisition by the second camera.

[0018] After the fourth application is allocated the data, a fourth interface is displayed. The fourth interface includes a floating window of the first application and images captured in real time by the second camera. In the fourth interface, the floating window of the first application includes images captured in real time by the first camera.

[0019] In the above scheme, during the traffic allocation stage corresponding to the fourth application, the terminal library reduces the processor resource requirements of the second camera by lowering the parameter values ​​of the settings in the driver settings, thereby enabling the first and fourth applications to access different cameras in parallel.

[0020] In one possible implementation of the first aspect, during the process of allocating traffic to the fourth application, the terminal can determine whether there are processor resources that support the operation of the second camera based on the third driver setting parameters selected for the second camera; if there are no processor resources that support the operation of the second camera, the terminal adjusts the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameters.

[0021] In the above scheme, during the traffic allocation stage corresponding to the fourth application, the terminal can accurately determine whether there are processor resources available to support the operation of the second camera based on the third driver setting parameters actually selected for the second camera. Furthermore, if it is determined that there are no processor resources available to support the operation of the second camera, the terminal reduces the processor resource requirements of the second camera by lowering the value of the first parameter corresponding to the first setting item in the third driver setting parameters, thereby enabling the first and fourth applications to access different cameras in parallel. In addition, relying on accurate judgment results, processor resources can be utilized more rationally while enabling the first and fourth applications to access different cameras in parallel.

[0022] In one possible implementation of the first aspect, the first setting includes the frame rate; the parameter value of the first setting includes a numerical value of the frame rate.

[0023] In the above solution, the processor resource requirements of the camera can be reduced by lowering the frame rate. For example, when the system camera and other applications access different cameras in parallel, the frame rate of the camera that the system camera wants to access can be reduced, thereby reducing the processor resources used by the system camera when accessing the camera, allowing the system camera and other applications to access different cameras in parallel.

[0024] Furthermore, reducing the frame rate minimizes the impact on user experience by reducing the camera's processor resource usage, as a slight reduction in frame rate has little effect on the user's visual perception. Moreover, reducing the frame rate does not affect the quality of individual images, thus achieving the effect of reducing ISP processor resource usage while maintaining image quality.

[0025] In one possible implementation of the first aspect, during the process of opening the second camera, the terminal can obtain resource usage information corresponding to the second application and determine whether there are processor resources available to support the operation of the second camera based on the resource usage information. The resource usage information is used to characterize the processor usage of the second camera, as instructed by the second application, during operation. If there are currently no processor resources available to support the operation of the second camera, during the traffic allocation phase corresponding to the second application, the terminal can adjust the third parameter value corresponding to the first setting item in the fifth driver setting parameters selected for the second camera to obtain a sixth driver setting parameter. The first setting item in the sixth driver setting parameter corresponds to a fourth parameter value; the fourth parameter value is less than the third parameter value; the sixth driver setting parameter is then issued to the second camera; the sixth driver setting parameter is used to control the second camera to acquire images.

[0026] In the above scheme, the terminal can obtain the resource usage information of the application during the OpenCamera stage to determine whether there are processor resources available to support the operation of the second camera. If it is determined that there are no processor resources available to support the operation of the second camera, during the traffic allocation stage of the second application, the terminal can reduce the processor resource requirements of the second camera by lowering the parameter value of the first setting item in the actually selected driver settings. This allows multiple applications to access different cameras in parallel, for example, enabling the first application and the second application to access different cameras in parallel.

[0027] In one possible implementation of the first aspect, during the process of opening the second camera, such as in the OpenCamera phase of the second application, the resource usage information acquired by the terminal is the first resource usage information. This first resource usage information is the resource usage information predicted during the terminal's initialization phase based on the target driver setting parameters corresponding to the second camera. The target driver setting parameters are the set of driver setting parameters that consumes the most processor resources among the multiple sets of driver setting parameters corresponding to the second camera.

[0028] In the above scheme, during the terminal initialization phase, the terminal predicts the first resource usage information corresponding to the second camera when it is called by the second application based on the set of driver settings parameters corresponding to the second camera that consumes the most processor resources. During the OpenCamera phase of the application, the terminal can determine whether there are processor resources available to support the operation of the second camera based on the first resource usage information predicted during the terminal initialization phase. Therefore, it can handle the situation accordingly based on the current processor resource usage. For example, if it is determined that there are no processor resources available to support the operation of the second camera, during the traffic allocation phase of the second application, the parameter value corresponding to the first setting item in the actually selected driver settings can be lowered, allowing multiple applications to access different cameras in parallel.

[0029] In one possible implementation of the first aspect, during the process of opening the second camera, i.e., the OpenCamera phase of the second application, the resource usage information obtained by the terminal is the second resource usage information; this second resource usage information is the resource usage information recorded in the whitelist for the second application. It should be understood that the second resource usage information recorded in the whitelist for the second application is a set empirical value. Compared to the resource usage information predicted based on the set of driver settings parameters corresponding to the second camera that have the highest processor resource usage, the resource usage information recorded in the whitelist is closer to the actual resource usage information corresponding to the second application when accessing the camera. Therefore, based on the second resource usage information recorded in the whitelist, it is possible to more accurately determine whether there are processor resources available to support the operation of the second camera.

[0030] In one possible implementation of the first aspect, if the application identifier of the second application is recorded in the whitelist, the resource usage information obtained by the terminal during the process of opening the second camera is the first resource usage information. If the application identifier of the second application is not recorded in the whitelist, the resource usage information obtained by the terminal during the process of opening the second camera is the second resource usage information. It should be understood that if the application identifier of the second application is recorded in the whitelist, it means that the second application is a whitelisted application; if the application identifier of the second application is not recorded in the whitelist, it means that the second application is not a whitelisted application. In the above scheme, the corresponding resource usage information is determined differently for whitelisted and non-whitelisted applications. The resource usage information corresponding to whitelisted applications is pre-recorded in the whitelist, and the resource usage information corresponding to whitelisted applications is closer to the resource usage information corresponding to the actual access to the camera. The resource usage information corresponding to non-whitelisted applications is determined during the terminal initialization phase based on the set of driver settings parameters with the highest resource usage. Therefore, the resource usage information corresponding to non-whitelisted applications often differs significantly from the resource usage information corresponding to the actual access to the camera. Therefore, compared to non-whitelisted applications, it can reduce the occurrence of situations where processor resources originally supported whitelisted applications accessing the camera, but were mistakenly judged as not supporting their access. To some extent, it is equivalent to giving whitelisted applications priority in resource allocation, ensuring that they can access the camera more stably than non-whitelisted applications.

[0031] In one possible implementation of the first aspect, the second application is an application recorded in a whitelist; the whitelist records the second resource usage information corresponding to the second application. During the process of opening the second camera, the system determines whether processor resources supporting the operation of the second camera exist based on the second resource usage information in the whitelist. If processor resources supporting the operation of the second camera exist, the terminal can display the image captured in real-time by the first camera in a floating window of the first application during the process of opening the second camera.

[0032] In the above scheme, a whitelist is set up, which records applications that have priority in using processor resources and their corresponding resource usage information. If the second application is on the whitelist (i.e., belongs to the whitelisted applications), then during the OpenCamera phase of the second application, the terminal can quickly and accurately determine whether there are processor resources available to support the operation of the second camera based on the second resource usage information recorded in the whitelist. If processor resources available to support the operation of the second camera are available, the first application's access to the first camera is maintained during the OpenCamera phase of the second application. Subsequently, the first application and the second application can access different cameras in parallel. Because the second resource usage information recorded in the whitelist is more accurate, it can reduce misjudgments of applications on the whitelist to a certain extent, that is, it can reduce the situation where applications on the whitelist are mistakenly judged as not supporting access to the camera when processor resources originally support access to the camera by the whitelist.

[0033] In one possible implementation of the first aspect, if, during the process of opening the second camera, it is determined, based on the second resource occupancy information corresponding to the second application recorded in the whitelist, that there are currently no processor resources supporting the operation of the second camera, then, during the process of opening the second camera, the display of the image captured by the first camera in the floating window of the first application is stopped, or a still image is displayed in the floating window of the first application.

[0034] In the above scheme, during the OpenCamera phase corresponding to the second application, the terminal can quickly and accurately determine whether there are processor resources available to support the operation of the second camera based on the second resource occupancy information recorded in the whitelist. If there are no processor resources available to support the operation of the second camera, the first application can be evicted from the camera during the OpenCamera phase corresponding to the second application, thereby freeing up processor resources for the second application to run. Because the second resource occupancy information recorded in the whitelist is more accurate, misjudgments can be avoided, and to some extent, the situation where camera access sessions of some applications are closed or disconnected due to misjudgment when multiple applications concurrently access different physical cameras can be reduced to some extent.

[0035] In one possible implementation of the first aspect, the terminal has a whitelist, and the second application is an application recorded in the whitelist (i.e., the second application belongs to the whitelist). If, during the process of opening the second camera, it is determined that there are currently no processor resources available to support the operation of the second camera, the terminal can stop displaying the image captured by the first camera in the floating window of the first application, or display a still image in the floating window of the first application. That is, it stops the first application's access to the first camera, releases processor resources, and thus allows for bandwidth allocation and subsequent processing of the second application.

[0036] In the above solution, applications on the whitelist are given priority in resource allocation to ensure that they can stably access camera resources.

[0037] In one possible implementation of the first aspect, during the process of opening the second camera, if the current usage scenario of the second application is a preset usage scenario, it is determined whether there are processor resources that support the operation of the second camera based on the second resource occupancy information in the whitelist.

[0038] In the above solution, a whitelist is set up in the terminal. The whitelist records applications that have priority in using processor resources and their corresponding resource usage information in preset usage scenarios. When the second application is on the whitelist and its current usage scenario matches the preset scenario, the terminal can accurately and conveniently determine whether there are processor resources available to support the second camera's operation based on the resource usage information recorded for the second application in the whitelist. This reduces false alarms to some extent, ensuring priority access to the camera for critical applications in specific usage scenarios, reducing camera screen freezes or interruptions caused by unreasonable resource allocation, and improving the user experience.

[0039] Secondly, this application provides a terminal that includes at least: multiple cameras, a memory, and one or more processors. The multiple cameras are used to capture images, and the memory is used to store computer instructions. When the one or more processors execute the computer instructions, the terminal performs the method described in any of the first aspects above.

[0040] Thirdly, this application provides a chip system applied to a terminal, the chip system including one or more processors, the processors being used to invoke computer instructions to cause the terminal to perform any of the methods described in the first aspect above.

[0041] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on a terminal, cause the terminal to perform any of the methods described in the first aspect above.

[0042] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform any of the methods described in the first aspect above. Attached Figure Description

[0043] Figure 1 This is a schematic diagram illustrating the relationship between the logical camera and the physical camera provided in an embodiment of this application.

[0044] Figure 2 This diagram illustrates the differences in ISP processor usage across different usage scenarios provided in the embodiments of this application.

[0045] Figures 3 to 4 A schematic diagram illustrating an application scenario where multiple applications access different physical cameras in parallel, as provided in an embodiment of this application.

[0046] Figure 5 A schematic diagram illustrating the failure of some applications to access different physical cameras in parallel when multiple applications access them in an embodiment of this application.

[0047] Figure 6 A schematic diagram illustrating the successful parallel access of multiple applications to different physical cameras provided in this application embodiment;

[0048] Figure 7 A hardware structure block diagram of a mobile phone provided in an embodiment of this application;

[0049] Figure 8 A software structure block diagram of a mobile phone provided in an embodiment of this application;

[0050] Figure 9 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 ;

[0051] Figure 10 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 2 ;

[0052] Figure 11 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 3 . Detailed Implementation

[0053] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0054] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. The terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature.

[0055] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0056] This application provides an image processing method applied to a terminal that has multiple applications installed and multiple physical cameras configured. This image processing method aims to improve the ability of multiple applications on the terminal to access different physical cameras in parallel, thereby reducing the occurrence of access failures for some applications in scenarios where multiple applications access different physical cameras in parallel (also referred to as "parallel access scenarios"). For example, in parallel access scenarios, it reduces the occurrence of camera screen freezes or interruptions for some applications.

[0057] A physical camera, also known as a physical camera, is an image acquisition device with physical hardware. That is, a physical camera is an actual, existing hardware device. For example, a physical camera includes hardware components such as a lens, image sensor, shutter, and aperture.

[0058] Taking a mobile phone as an example, suppose the phone includes one front-facing camera and three rear-facing cameras. The three rear-facing cameras can include a main camera, a wide-angle camera, and a telephoto camera. All four cameras mentioned above are physical cameras.

[0059] It should be understood that the physical camera is the underlying hardware, and the upper-level software of the terminal (such as applications) generally accesses or calls the underlying physical camera by accessing or calling the abstract logical camera.

[0060] A logical camera, also known as a logical camera, is an abstract device corresponding to a physical camera. The upper-layer software (such as applications) on the terminal manages the physical camera by invoking the logical camera. In other words, the logical camera is an abstract camera that is known to the upper-layer software. A logical camera can also be called a virtual camera.

[0061] One logical camera can correspond to one or more physical cameras. Each logical camera is primarily used to manage and control the corresponding physical camera. By simultaneously controlling the physical cameras corresponding to the logical cameras, more complex image processing and shooting effects can be achieved.

[0062] For example, logical cameras are primarily used in scenarios requiring the collaboration of physical cameras. Logical cameras can achieve better shooting results by integrating images captured by multiple physical cameras. Taking a mobile phone as an example, one logical camera can correspond to the wide-angle camera and telephoto camera in the phone. The upper-level software can call this logical camera to control the corresponding wide-angle and telephoto cameras to achieve functions such as optical zoom and background blur.

[0063] To make it easier to understand, let's take a mobile phone with four physical cameras as an example, combined with... Figure 1 This diagram illustrates the relationship between physical cameras and logical cameras. Please refer to [link / reference]. Figure 1The phone has four physical cameras: a front-facing camera (101), a main camera (102), a telephoto camera (103), and a wide-angle camera (104). The main camera (102), telephoto camera (103), and wide-angle camera (104) are rear cameras. The phone also has three logical cameras: logical camera 0, logical camera 1, and logical camera 2.

[0064] Logical camera 0 can correspond to front-facing camera 101. The application can call logical camera 0 to control front-facing camera 101 to achieve image processing in various scenarios such as video calls, selfies, live streaming, and face recognition.

[0065] Logical camera 2 can correspond to main camera 102 and telephoto camera 103. The application can call logical camera 2 to control main camera 102 and telephoto camera 103 to capture more distant and finer scene details, and can achieve background blur effect.

[0066] Logical camera 1 can correspond to main camera 102 and wide-angle camera 104. The application can call logical camera 1 to control main camera 102 and wide-angle camera 104, providing a wider field of view when shooting landscapes, large events, or other scenes that need to include more elements.

[0067] The applications or applications (e.g., the first application, the second application, the third application, the fourth application, etc.) in this application embodiment refer to applications with camera access permissions or camera call permissions, such as the system camera (also called the camera application) and third-party applications with camera access permissions. These applications are upper-layer applications in the software system, which can call the logical camera and access the underlying physical camera, thereby controlling the physical camera to perform image acquisition and processing in different usage scenarios.

[0068] It should be understood that when a logical camera is invoked or accessed by an application, it needs to rely on or occupy processor resources (such as the ISP processor) to fulfill the corresponding image processing requirements. The camera's resource usage information is used to characterize the processor resource usage when the logical camera is accessed or used by the application. For example, it is used to characterize the usage of the ISP processor (Image Signal Processor).

[0069] In some embodiments, camera resource usage information may include a resource usage value. In some examples, the resource usage value may be the ratio of the number of ISP processors required when the logical camera is invoked by an application to the total number of ISP processors in the terminal. For example, if the terminal has a total of 3 ISP processors, and logical camera 1 is invoked by an instant messaging application for a video call, requiring 1 ISP processor, then in this video call scenario, the resource usage value corresponding to logical camera 1 may be 1 / 3 * 100, approximately equal to 33%. In other examples, the resource usage value may also be directly the number of ISP processors required when the logical camera is invoked by an application, without limitation.

[0070] In other examples, the resource usage value can also be the number of ISP processors required when a logical camera is invoked by an application. For example, when logical camera 1 is invoked by an instant messaging application for a video call, it requires 1 ISP processor. Therefore, the resource usage value corresponding to logical camera 1 can be 1.

[0071] Resource usage information can also be other information that can characterize resource usage, and there are no restrictions on this.

[0072] It should be understood that a single logical camera can correspond to multiple usage scenarios. In this application's embodiments, the usage scenario refers to the scenario in which the logical camera or physical camera is used or accessed. For example, video call scenarios, photo-taking scenarios, video recording scenarios, and live streaming scenarios are all different usage scenarios.

[0073] A logical camera can be accessed by different applications in various usage scenarios to control the corresponding physical camera to capture and process images, meeting the image processing needs of different scenarios. The logical camera relies on processor resources (e.g., the ISP processor) to meet the image processing requirements of various usage scenarios. Since the image processing requirements may differ in different usage scenarios, the processor resource (e.g., ISP processor) usage for the same logical camera will also vary. That is, the resource usage information for the same logical camera may differ in different usage scenarios. It should be understood that since the logical camera's processor resource usage occurs when it is accessed by an application, the logical camera's resource usage information can also be referred to as the resource usage information of the application currently accessing the logical camera. In other words, the resource usage information of the logical camera in the current usage scenario can also be referred to as the resource usage information of the application accessing the logical camera in the current usage scenario.

[0074] The following text, using Examples 1 and 2, illustrates the differences in ISP processor usage across various use cases.

[0075] Example 1: When using different display modes for dual-view recording, the ISP processor usage of the logical camera will vary.

[0076] Let's denote the scene using a split-screen display mode in dual-view recording as Scene 1, and the scene using a floating window display mode in dual-view recording as Scene 2. Assume that in both Scene 1 and Scene 2, it's necessary to call logical camera 1, which controls its corresponding main camera and telephoto camera to capture images and perform corresponding image processing.

[0077] Please see Figure 2 In scenario (a), images captured by different cameras are displayed in a split-screen format. The upper screen 201 displays images captured by the main camera, such as people indoors. The lower screen 202 displays images captured by the telephoto camera, such as trees in the distance outdoors.

[0078] Please see Figure 2 In scenario (b), the main screen 203 displays the image captured by the main camera, while the floating window 204 displays the image from the telephoto camera.

[0079] By comparison Figure 2 (a) and Figure 2 As shown in (b), although the phone calls or accesses the main camera and telephoto camera corresponding to the same logical camera 1 for image acquisition in both scenario 1 and scenario 2, the image processing requirements of scenario 1 and scenario 2 are different. For example, in the same display mode, the image size acquired by the same main camera or the same telephoto camera is different. Therefore, the driver settings (also known as "Sensor Setting") of the same physical camera 1 (main camera or telephoto camera) will differ in different scenarios, and the ISP processor usage of logical camera 1 will also differ.

[0080] Example 2: The ISP processor usage of the logical camera differs between the dual-view recording mode and the portrait mode.

[0081] Specifically, when the system camera is in portrait mode, it can also call logical camera 1 to control the main camera and telephoto camera corresponding to logical camera 1 to acquire and process images, thereby displaying a portrait preview image in the preview stream of the camera's shooting interface. The image acquired by the telephoto camera is used for blurring and is not displayed, while the image acquired by the main camera is used for preview display in the shooting interface.

[0082] For telephoto cameras, the scenarios are completely different: blurring and not displaying images captured in portrait mode versus displaying the captured images in dual-view recording mode. The SensorSettings of the telephoto camera will differ in these two scenarios, the image acquisition requirements will differ, and therefore the ISP processor usage will also differ.

[0083] It should be understood that a logical camera is an abstract device and does not possess actual image acquisition capabilities. Therefore, the driver settings parameters for a logical camera are primarily for the sensor driver settings of a physical camera. The following section will describe these driver settings parameters in detail.

[0084] Driver settings parameters refer to the settings parameters of the physical camera's sensor driver. The sensor driver refers to the driver of the image sensor. For example, driver settings parameters may include parameters such as resolution and frame rate. It should be understood that driver settings parameters may also include other parameters, which are not exhaustive here.

[0085] As mentioned above, a logical camera has a corresponding physical camera. For the same logical camera, the image acquisition requirements of the corresponding physical camera differ in different usage scenarios, therefore, the corresponding Sensor Settings will also differ. That is, a logical camera can correspond to multiple sets of SensorSettings (i.e., multiple sets of driver settings parameters) to meet the image acquisition needs of different usage scenarios. Each set of Sensor Settings controls the image sensor in the physical camera to achieve image acquisition and processing for the corresponding usage scenario. The driver settings parameters correspond to the current usage scenario of the terminal or the camera function that needs to be implemented.

[0086] As mentioned above, different sets of Sensor Settings can meet different usage scenarios and image acquisition requirements. Therefore, the resource requirements of the Image Signal Processor (ISP) may differ. In other words, for multiple sets of Sensor Settings corresponding to the same logical camera, each set of Sensor Settings requires different processor resources, such as the number of ISP processors it occupies.

[0087] Similarly, referring to Example 1 above... Figure 2 (a) and Figure 2 (b) is illustrated.

[0088] Assume logical camera 1 corresponds to N sets of Sensor Settings, including Settings1 and Settings2. Settings1 includes the resolution r11 and frame rate f11 of the main camera, and the resolution r21 and frame rate f21 of the telephoto camera. Settings2 includes the resolution r12 and frame rate f12 of the main camera, and the resolution r22 and frame rate f22 of the telephoto camera.

[0089] The main camera's resolution r11 and resolution r12 may be different, or the main camera's frame rate f11 and frame rate f12 may be different. Similarly, the telephoto camera's resolution r21 and resolution r22 may be different, or the telephoto camera's frame rate f21 and frame rate f12 may be different.

[0090] exist Figure 2 In scenario 1 shown in (a), when camera1 uses Settings1 to control the corresponding physical camera for image acquisition, it may require 3 ISP processors. Figure 2 In scenario 2 shown in (b), when camera1 uses Settings2 to control the corresponding physical camera for image acquisition, it may require two ISP processors. That is, in different scenarios, the driver settings parameters for the same logical camera may be different, and therefore the ISP processor usage will also be different.

[0091] It should be understood that the terminal startup process includes an initialization phase that reports camera capabilities. After the initialization phase, applications can request access to the camera or webcam. The process of an application accessing the webcam includes the OpenCamera phase and the Configure_stream phase.

[0092] The initialization phase, also known as the init phase or the boot phase, refers to the initialization phase of the terminal itself or the entire device. The initialization phase involves the initialization processing related to the camera or webcam, specifically including the terminal reporting camera capabilities during the boot phase.

[0093] It should be understood that the initialization phase is unrelated to the application's camera access processing, which is a series of initialization processes performed during the boot phase before the application requests access to the camera or webcam.

[0094] After the initialization phase, applications on the terminal with camera access permissions or functions can respond to application A's request to open the camera by calling the `Open()` function or the `open` interface to initiate a camera access request. The terminal can then determine the target logical camera requested in the request and enter the `OpenCamera` phase corresponding to application A. The target logical camera refers to the logical camera requested in this access request. The `OpenCamera` phase for application A is primarily used to establish the camera access path. It should be understood that the `OpenCamera` phase establishes the camera access path at the software level, allowing subsequent access to the underlying physical camera.

[0095] After establishing the camera access path, the Configure_stream stage corresponding to application A can be entered. Here, stream allocation is performed for application A, thereby powering on and initializing the physical camera requested by application A in this camera access request. It should be understood that during the processing of application A's camera access request, both the logical camera accessed and the physical camera ultimately invoked at the underlying level belong to the camera requested in the camera access request.

[0096] Specifically, in the Configure_stream phase, the application can call the Configure_stream() function or the Configure_stream interface to configure the stream.

[0097] As mentioned above, the logical camera is an abstract device. The driver settings (SensorSettings) for the logical camera mainly refer to the sensor driver settings for the physical camera. The physical camera primarily acquires images based on these Sensor Settings. Therefore, in the Configure_stream phase, the terminal determines the actual Sensor Settings selected for the target logical camera based on the streaming information corresponding to this camera access request. Then, the terminal writes the actually selected Sensor Settings into the registers of the physical camera corresponding to the target logical camera, thereby performing power-on initialization of the physical camera based on the written Sensor Settings.

[0098] The actual selected Sensor Settings refer to the Sensor Settings selected from the multiple sets of Sensor Settings included in the target logical camera, which are the Sensor Settings required for this camera access request.

[0099] After powering on the physical camera during the Configure_stream phase, the upper-layer application can request to start the physical camera to capture image streams. It should be understood that the target logical camera may correspond to one or more physical cameras; therefore, the physical camera corresponding to the target logical camera can also be referred to as a physical camera module.

[0100] As mentioned earlier, traditional methods typically cannot handle situations where multiple applications access different physical cameras in parallel. Specifically, when multiple applications access different physical cameras in parallel, some applications often fail to access their corresponding cameras correctly. For example, if a second application accesses camera 2 in parallel while the first application is accessing camera 1, the first application may be unable to access camera 1 correctly.

[0101] For ease of understanding, the following text uses a mobile phone as an example, combined with the attached... Figure 3 To be continued Figure 5 This illustration shows an application scenario where multiple applications cannot access different physical cameras in parallel.

[0102] Please see Figure 3 (a) The mobile phone displays a video call interface 310 via a third-party application. This video call interface 310 includes a local image 311 and a remote image 312. The local image 311 is an image captured by the phone's front-facing camera. During the video call, if the mobile phone user (i.e., the user of the mobile phone) wants to take a photo while making the video call, they can click icon 313 to zoom out of the video call interface 310. Figure 3 As shown in (b), the mobile phone can display the conversation list interface 320 of the third-party application. The floating window 321 corresponding to the third-party application is displayed in the conversation list interface 320. The image content of both parties in the video call will be shrunk and displayed in the floating window 321.

[0103] Mobile phone users can interact with the conversation list interface 320, triggering the phone to return to the home screen 330. Specifically, as follows... Figure 4 As shown in (a), the desktop 330 displays a camera icon (i.e., the system camera icon) 331 and a floating window 321 of a third application. The floating window 321 displays the image content of both parties in the video call; for example, the floating window 321 includes the local image 3211. It should be understood that... Figure 4 The local end image 3211 in (a) and Figure 3 In (a), the local image 311 is an image captured for the same user on the local device (e.g., a user using the mobile phone). The mobile phone can respond to the user's triggering action on the camera icon 331 and open the system camera.

[0104] Please see Figure 4(b) After the system camera is opened, the phone can display interface 340. This interface 340 includes a floating window 321 and a camera preview interface 341 for the system camera. The camera preview interface 341 shows the image obtained by the system camera through accessing the phone's rear camera. In this way, the system camera and third-party applications access different cameras in parallel, namely the front camera and the rear camera.

[0105] like Figure 4 As shown in (b), 3212 is the account avatar of the local user (i.e., the user logging into the third-party application using their mobile phone) in the third-party application, not the local image. Clearly, when the system camera and the third-party application access the front and rear cameras in parallel, respectively, it causes an anomaly in the third-party application's access to the front camera, preventing it from capturing and displaying the local image during the video call. That is, for example, opening the system camera during a video call will cause the user's image on the mobile phone to be undisplayed in the original video call interface. It should be understood that although... Figure 4 (b) in the text does not illustrate the camera screen freezing problem caused by abnormal camera access. However, in reality, in scenarios where multiple applications access different cameras in parallel, in addition to causing some applications' camera screens not to be displayed, it may also cause some applications' camera screens to freeze.

[0106] To better understand the situation where some applications cannot access their corresponding cameras properly when multiple applications access different physical cameras in parallel, Figure 5 This is an illustration from the perspective of the other end of the video call. The other end of the video call is the device that is making a video call with the mobile phone.

[0107] Please see Figure 5 (a) Assuming the local device is a mobile phone, if the third-party application that initiates the video call on the mobile phone can normally access the mobile phone's physical camera (such as the front camera), the other end of the video call will display a video call interface 510, where 511 is the image captured by the third-party application on the mobile phone by accessing the physical camera.

[0108] like Figure 4 (a) and Figure 4 As shown in (b), during a video call using a third-party application on the phone, if the phone's system camera is open, the other end of the video call will see the video call interface 520, specifically as follows: Figure 5(b) The video call interface 520 no longer displays screen 511, but only the avatar 521 of the account logged into the third-party application on the phone. This is because, during the video call process when the third-party application accesses the phone's front camera, the system camera simultaneously accesses the phone's rear camera, preventing the third-party application from properly accessing the phone's front camera for image capture.

[0109] In one approach (referred to as Approach 1), during the terminal initialization phase, the terminal can predict the resource usage information for each logical camera based on the Sensor Settings group with the highest resource consumption among the Sensor Settings groups included in that logical camera. Application A in the terminal can respond to the user's camera-opening operation by calling the Open() function to initiate this camera access request and enter the OpenCamera phase corresponding to application A.

[0110] During the OpenCamera phase corresponding to application A, the terminal can determine whether there are processor resources available to support the operation of the camera (logical or physical camera) requested by application A, based on the resource usage information predicted during the initialization phase. The terminal can then perform corresponding camera control access processing based on this result. It should be understood that determining whether processor resources exist to support the operation of the camera requested by application A is equivalent to determining whether the current processor resources support multiple applications accessing different cameras in parallel.

[0111] Combination Figure 4 For example, application A could be... Figure 4 The system camera represented by camera icon 331 in (a). In response to Figure 4 (a) After the system camera (i.e. the system's camera application) is opened, in the OpenCamera phase of the system camera, it is determined whether the current processor resources support the system camera and third-party applications to access the front and rear cameras in parallel.

[0112] It should be understood that the judgment result can include Result 1 or Result 2. Result 1 indicates that there are currently processor resources available to support the operation of the camera (logical camera or physical camera) requested by application A (i.e., the current processor resources support multiple applications accessing different cameras in parallel). Result 2 indicates that there are currently no processor resources available to support the operation of the camera (logical camera or physical camera) requested by application A (i.e., the current processor resources do not support multiple applications accessing different cameras in parallel).

[0113] If result 1 is obtained, the terminal can establish a camera access path for this camera access request, enter the Configure_stream stage corresponding to application A, allocate the stream for application A, power on and initialize the corresponding physical camera, and perform subsequent image acquisition and processing. This allows multiple applications to access different cameras in parallel. See [link to details] for more information. Figure 6 (b) The interface 620 can display both the image 6111 captured by the front camera called by the video call application and the image 621 captured by the rear camera called by the system camera.

[0114] If result 2 is obtained, the terminal can evict other applications from camera access during the OpenCamera stage corresponding to application A, thereby freeing up ISP processor resources for application A's current camera access request. That is, the phone can then establish a camera access path for this request and enter the Configure_stream stage to allocate streams for application A. In this case, multiple applications failing to access different cameras in parallel will result in failure; see [link to documentation] for details. Figure 4 (b)

[0115] Camera expulsion refers to the system expelling camera access permissions for certain applications or applications due to resource consumption limitations (mainly ISP processor resource limitations). When the available processor resources (such as ISP processor resources) are insufficient, the system will expel these permissions to free up ISP processor resources.

[0116] Revoking camera access for certain applications or specific applications is equivalent to shutting down the physical camera corresponding to the logical camera accessed by those applications. In the event of camera revocation, the application's camera view may freeze (i.e., the image stops changing and remains static) or become interrupted (i.e., it stops displaying).

[0117] In the above scheme 1, when the application initiates a camera access request, it determines whether the current processor resources support multiple applications accessing different cameras in parallel based on the resource usage information of the logical camera predicted during the terminal initialization phase. If supported, multiple applications can access different cameras in parallel.

[0118] In some embodiments, the terminal may display a first interface including a floating window of a first application, the floating window including images captured in real time by a first camera. In response to opening a second application, a second camera is opened. Opening the camera is equivalent to Opencamera; specifically, the terminal may call the Open() function to open the second camera.

[0119] While the second camera is being opened, the real-time images captured by the first camera are still displayed in the floating window of the first application. That is, during the Opencamera process, the first application can still access the camera.

[0120] After the Opencamera phase for the second application, the streaming phase for the second application begins, where the terminal streams the second application. For example, the terminal can use the Configure_stream() function to stream the second application.

[0121] After the second application is streamed, a second interface is displayed. The second interface includes images captured in real time by the second camera and a floating window of the first application, which includes images captured in real time by the first camera.

[0122] It should be understood that the first camera, second camera, and other cameras in the embodiments of this application can be logical cameras or physical cameras. A logical camera can also be called a virtual camera, and logical cameras correspond to physical cameras. As mentioned above, one logical camera can correspond to one or more physical cameras.

[0123] To facilitate understanding, we will now combine Figure 6 The following is an illustrative explanation. The terminal can display a first interface, such as... Figure 6 The desktop 610 is shown in (a). Desktop 610 includes a floating window 611 for a video call application, which displays an image captured by the terminal's front-facing camera. When the user clicks the system camera icon 612, the terminal responds by opening the system camera and the rear camera. During the opening of the rear camera, the image captured by the front-facing camera is displayed in the floating window 611 of the video call application. The terminal can allocate the system camera stream, and after allocating the system camera stream, a second interface 620 is displayed, such as... Figure 6 As shown in (b). The second interface 620 includes a floating window 611 for the video call application and an image 621 captured by the rear camera. In the second interface 620, the floating window 611 for the video call application includes an image 6111 captured by the front camera.

[0124] By comparison Figure 6 (b) and Figure 4 As can be seen from (b), the method in the above embodiments enables the first application and the second application, such as the camera application and the video call application, to access the camera normally in parallel. That is, it enables multiple applications to access or successfully access different cameras in parallel.

[0125] In some embodiments, the terminal allocates traffic to the second application by: determining whether there are processor resources that support the operation of the second camera based on the first driver setting parameters selected for the second camera; if there are processor resources that support the operation of the second camera, issuing the first driver setting parameters to the second camera; the first driver setting parameters are used to control the second camera to acquire images.

[0126] Specifically, in this embodiment, during the Configure_stream stage of the second application, the terminal can obtain the Sensor Settings actually selected for the second camera, i.e., the first driver setting parameters. Then, during the stream allocation process for the second application, the terminal can determine whether there are processor resources available to support the operation of the second camera requested by the second application, based on the actually selected Sensor Settings, and obtain a determination result. It should be understood that the obtained determination result can also include result 1 or result 2.

[0127] If there are processor resources available to support the operation of the second camera requested by the second application (i.e., result 1 is obtained), the terminal can issue the first driver setting parameters to the second camera. After the second application's streaming is completed, during the process of starting the image acquisition stream, the second camera can acquire images based on the first driver setting parameters. Therefore, after streaming the second application, a second interface is displayed. The second interface includes a floating window for the first application and the real-time image acquired by the second camera. In the second interface, the floating window for the first application includes the real-time image acquired by the first camera. This real-time image acquired by the second camera is the image acquired by the second camera based on the first driver setting parameters.

[0128] In some examples, if there are currently no processor resources available to support the operation of the second camera requested by the second application (i.e., result 2 is obtained), the terminal can perform camera eviction on the first application during the Configure_stream stage corresponding to the second application. This releases the processor resources (such as ISP processor resources) occupied by the first application when using the first camera, and allows the released processor resources to be used by the second application to run the second camera. In this way, the stream allocation processing during the Configure_stream stage can be completed for the second application. Because the camera is evicted from the first application during the Configure_stream stage corresponding to the second application, the interface displayed after stream allocation for the second application includes the real-time image captured by the second camera and the floating window of the first application, but the floating window of the first application does not include the image captured by the first camera.

[0129] In other words, in this embodiment, after receiving a camera access request from an application, the terminal no longer predicts the resource usage information corresponding to the logical camera during the initialization phase, nor does it determine whether there are processor resources available to support the operation of the camera requested by the application during the OpenCamera phase of the application. Instead, it makes the above determination based on the actually selected Sensor Settings during the application's flow allocation phase. Figure 6 Let's illustrate with examples. Figure 6 As shown in (a), after the user triggers the camera icon 612 of the system camera (i.e., the system's camera application), the system enters the OpenCamera phase corresponding to the system camera. The terminal does not determine whether there are processor resources that support the operation of the camera requested by the system camera in the OpenCamera phase corresponding to the system camera, but performs the above determination in the Configure_stream phase corresponding to the system camera.

[0130] The steps for the terminal to allocate traffic to the second application are not limited to the steps mentioned above. They may also include other steps, such as obtaining traffic allocation information and selecting appropriate driver settings parameters for the second camera based on the traffic allocation information.

[0131] It should be understood that compared to predicting the resource usage of the logical camera based on the Sensor Settings with the highest resource consumption during the initialization phase, the actually selected Sensor Settings more accurately reflect the resource usage of the second camera during operation. Furthermore, the resource usage corresponding to the actually selected Sensor Settings is highly likely to be less than the resource usage predicted based on the Sensor Settings with the highest resource consumption. Therefore, determining whether there are processor resources available to support camera operation based on the actually selected Sensor Settings yields a more accurate result. This prevents situations where resources are sufficient from being misjudged, or reduces the likelihood of misjudging as insufficient resources. Consequently, when multiple applications concurrently access different physical cameras, the camera access sessions of some applications may be closed or disconnected due to misjudgment, allowing multiple applications to successfully access different physical cameras in parallel in more scenarios.

[0132] In some embodiments, the terminal can exit the second application in response to a user operation and return to display the first interface. If the terminal receives an operation from the user to open a third application based on the first interface, the terminal can respond by opening the third application and opening the second camera. Entering the Opencamera phase corresponding to the third application, the third application also requests access to the second camera. During the process of opening the second camera, the first application can still access the first camera normally, and the real-time image captured by the first camera is displayed in the floating window of the first application.

[0133] After the Opencamera phase corresponding to the third application, the Configure_stream phase corresponding to the third application begins, where the third application is streamed. During the stream streaming process for the third application (i.e., the Configure_stream phase), the terminal stops displaying the image captured by the first camera in the floating window of the first application, or, during the stream streaming process for the third application, displays a still image in the floating window of the first application.

[0134] In this scenario, the first application is essentially "expelled" by the camera. It can no longer access the first camera, so its floating window no longer displays the image captured by the first camera. In other words, the floating window freezes, displaying only a static image instead of the real-time image. Once the first application can no longer access the first camera, the processor resources used by the first camera are released. These released resources can then be used by the third application to access the second camera, thus successfully allocating bandwidth to the third application.

[0135] After the third application is allocated the data, a third interface is displayed. The third interface includes the floating window of the first application and the image captured in real time by the second camera. In the third interface, the floating window of the first application does not include the image captured by the first camera, or the floating window of the first application displays a still image in the third interface.

[0136] In some embodiments, during the allocation of traffic to the third application, the terminal can obtain the second driver setting parameters actually selected for the second camera. These second driver setting parameters are automatically selected to meet the image acquisition requirements of the third application using the second camera; that is, the images acquired by the second camera using the second driver setting parameters can fully meet the image acquisition requirements of the third application. The terminal can determine whether there are processor resources available to support the operation of the second camera based on the second driver setting parameters selected for the second camera. The determination result can also include result 1 or result 2.

[0137] If there are currently no processor resources to support the operation of the second camera (i.e., result 2), then as mentioned above, the first application's access to the first camera will be stopped during the Configure_stream stage corresponding to the third application. Therefore, the image captured by the first camera will stop being displayed in the floating window of the first application. Alternatively, during the distribution of the stream to the third application, a still image will be displayed in the floating window of the first application.

[0138] In some embodiments, if processor resources supporting the operation of the second camera are available (i.e., result 1), the terminal can issue second driver setting parameters to the second camera. After the third application stream is allocated, during the process of starting the image acquisition stream, the second camera can acquire images based on the second driver setting parameters. Therefore, the interface displayed after the third application stream is allocated includes a floating window of the first application and images acquired in real time by the second camera based on the second driver setting parameters. The floating window of the first application includes the images acquired in real time by the first camera.

[0139] In some embodiments, the terminal may display a first interface, which may include a floating window of a first application, the floating window including an image captured in real time by a first camera. In response to opening a second application based on the first interface, the terminal may open a second camera, during which the image captured in real time by the first camera is displayed in the floating window of the first application. The terminal may stream a first shooting mode of the second application. After streaming the second application, the terminal may display a second interface, the second interface including the floating window of the first application and the image captured by the second camera, where the floating window of the first application includes the image captured by the first camera. In response to a first operation, the terminal may stream a second shooting mode of the second application on the second interface, stopping the display of the image captured by the first camera in the floating window of the first application during the streaming of the second shooting mode. After streaming the second shooting mode of the second application, the terminal may display a third interface, the third interface including the floating window of the first application and the image captured by the second camera, where the floating window of the first application does not include the image captured by the first camera.

[0140] The first operation can be the operation of switching modes.

[0141] In the above embodiments, the image processing requirements of the second application under the first shooting mode and the second shooting mode may be different, and therefore, the requirements for processor resources will also be different. Therefore, the flow distribution stage of the second application differs under different shooting modes. For example, after the flow distribution is completed under the first shooting mode, the first application can access the first camera. That is, the second interface displayed after the flow distribution of the second application under the first shooting mode includes the floating window of the first application and the image captured by the second camera. In the second interface, the floating window of the first application includes the image captured by the first camera.

[0142] During the streaming phase of the second shooting mode for the second application, the display of the image captured by the first camera in the floating window of the first application will be stopped. For example, in this case, the terminal can ban the camera from the first application, freeing up processor resources such as the ISP processor. Then, after streaming the second shooting mode for the second application, a third interface is displayed. The third interface includes the floating window of the first application and the image captured by the second camera. Compared to the second interface displayed after streaming the first shooting mode for the second application, the floating window of the first application in the third interface does not include the image captured by the first camera.

[0143] For example, the aforementioned terminal can be a mobile phone, tablet, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as a cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, or any other terminal with multiple physical cameras. This application does not impose special limitations on the specific form of the terminal. The following mainly uses a mobile phone as an example to illustrate the solution of this application. It should be understood that the principle of implementing the solution of this application in other terminals is the same as that in a mobile phone, therefore, it will not be described in detail for different types of terminals.

[0144] See Figure 7The mobile phone may include a processor 710, an external memory interface 720, an internal memory 721, a universal serial bus (USB) interface 730, a charging management module 740, a power management module 741, a battery 742, antenna 1, antenna 2, a mobile communication module 750, a wireless communication module 760, an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, a headphone jack 770D, a sensor 780, buttons 790, a motor 791, an indicator 792, a camera 793, a display screen 794, and a subscriber identification module (SIM) card interface 795, etc.

[0145] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the mobile phone. In other embodiments of this application, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0146] The processor 710 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0147] In some embodiments, the mobile phone can perform image processing through the processor 710, enabling multiple applications to access different cameras 793 in parallel in more scenarios, thereby reducing the occurrence of some applications being unable to access the corresponding camera 793 when multiple applications access different cameras 793 in parallel.

[0148] Camera 793 is an actual physical camera. In this embodiment, the mobile phone has multiple cameras 793, for example, multiple cameras 793 may include front-facing cameras and rear-facing cameras. There may be one or more front-facing cameras, and there may also be one or more rear-facing cameras.

[0149] A mobile phone can perform shooting functions through an ISP, camera 793, video codec, GPU, display 794, and application processor. For example, an application in the mobile phone can access the camera 793 and control it to capture images. The ISP provides hardware support for image acquisition and processing. In this embodiment, multiple applications in the mobile phone can access different cameras 793 in parallel.

[0150] The mobile phone implements its display function through a GPU, a display screen 794, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 794 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 710 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0151] The display screen 794 is used to display images, videos, etc. In some embodiments, the mobile phone can display the interfaces generated by different applications accessing the camera through the display screen 794. For example, the mobile phone can display the video call interface generated by an application accessing the front camera through the display screen 794. During a video call, the mobile phone can also display the interface generated by the system camera accessing the rear camera through the display screen 794.

[0152] Mobile phones can perform audio functions, such as music playback and recording, through components like the audio module 770, speaker 770A, receiver 770B, microphone 770C, headphone jack 770D, and application processor.

[0153] The mobile phone's wireless communication function can be implemented through antenna 1, antenna 2, mobile communication module 750, wireless communication module 760, modem processor, and baseband processor. For example, the mobile phone can establish a video call with a peer device through mobile communication module 750.

[0154] Buttons 790 may include a power button, volume buttons, etc. Buttons 790 can be mechanical buttons or touch buttons. The phone can receive button input and generate key signal inputs related to user settings and function control. Motor 791 can generate vibration alerts. Motor 791 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 792 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 795 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 795 to achieve contact and separation with the phone.

[0155] Mobile phone software systems can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses a layered architecture software system as an example to illustrate the software structure of a mobile phone. It should be noted that this application is not limited to the software structure of mobile phones.

[0156] Figure 8 A software structure block diagram illustrating the mobile phone provided in the embodiments of this application.

[0157] As we can understand, a layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 8 As shown, a software system may include an application layer (APP layer), an application framework layer (Framework layer), a hardware abstraction layer (HAL layer), and a kernel layer (also known as a driver layer).

[0158] The application layer can include a series of application packages, including system applications and third-party applications. System applications may include a gallery (not shown in the diagram), a system camera, etc. Third-party applications may include applications with camera access permissions or camera access functionality, such as third-party video applications (not shown in the diagram) and third-party instant messaging applications (not shown in the diagram).

[0159] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0160] In some embodiments of this application, the application framework layer may include a Camera Service and a camera access interface. The camera access interface provides the application with a standard API for accessing camera hardware (such as a webcam). The camera access interface may include an open interface, a Configure_stream interface, a close interface, etc. The open interface is used to open the camera; the Configure_stream interface is a streaming interface; and the close interface is used to close camera access.

[0161] The Hardware Abstraction Layer (HAL) is an interface layer located between the local service layer and the kernel layer, providing a virtual hardware platform for the operating system.

[0162] In some embodiments of this application, the hardware abstraction layer may include a camera hardware abstraction layer (Camera HAL). The camera hardware abstraction layer provides a direct control interface to the camera hardware.

[0163] The kernel layer is the layer between hardware and software. The kernel layer can include sensor drivers and display drivers.

[0164] The sensor driver includes the driver program for the image sensor in the camera, used to control the camera in the hardware layer to acquire images. For example, controlling... Figure 8 The image is captured by multiple cameras shown. One sensor driver can control multiple cameras. Alternatively, one sensor driver can control one camera; the relationship between the sensor driver and the camera is one-to-one. This is not a limitation.

[0165] The display driver is used to control the content displayed on the screen in the hardware layer. For example, in some embodiments of this application, the display driver is used to control the mobile phone to display a video call interface formed by images captured by the front-facing camera on the screen. During a video call, if the mobile phone responds to the user's operation and opens the system camera, the mobile phone can also display a camera interface on the screen, displaying a preview image captured by the rear-facing camera. Exemplarily, the video call interface can be displayed as a floating window on the camera interface.

[0166] It should be noted that, Figure 8 This is only used to illustrate the modules in a mobile phone's software system and does not imply that the mobile phone's software system must have or only have these modules.

[0167] The following sections will describe the solutions in the embodiments of this application in more detail with reference to the accompanying drawings.

[0168] As mentioned above, the terminal can predict the resource usage information for each logical camera based on the Sensor Settings group with the highest resource consumption among the Sensor Settings groups included in that logical camera. The following section will detail how to predict the resource usage information for each logical camera.

[0169] It should be understood that each logical camera can correspond to multiple sets of Sensor Settings, and each set of Sensor Settings consumes different amounts of processor resources. For ease of explanation, the amount of processor resources consumed by the set of Sensor Settings with the highest processor resource consumption (e.g., the amount of processor resources consumed by the ISP) can be simply referred to as the maximum resource consumption.

[0170] In some examples, during the initialization phase, the phone can determine the resource usage information corresponding to a logical camera based on the maximum resource usage. For instance, the phone can determine the resource usage information for a logical camera based on the ratio of the maximum resource usage to the total amount of processor resources. Alternatively, the phone can directly use the maximum resource usage as the resource usage information for the logical camera.

[0171] Taking the processor resource usage as the ISP usage as an example, assume that logical camera 1 corresponds to 2 SensorSettings, denoted as Settings1 and Settings2 respectively. Assume there are a total of 3 ISP processors in the phone. During the boot process, the phone can calculate that the ISP usage for Settings1 is 2 (meaning that using Settings1 to control the corresponding physical camera for image acquisition requires 2 ISP processors), and the ISP usage for Settings2 is 1 (meaning that using Settings2 to control the corresponding physical camera for image acquisition requires 1 ISP processor). Therefore, Settings1 corresponds to the maximum ISP usage. The phone can then calculate the resource usage value for logical camera 1 during boot based on the ratio of the maximum ISP usage value 2 (i.e., requiring 2 ISP processors) to the total number of ISP processors 3. For example, the resource usage value for logical camera 1 = 2 / 3 * 100 = 66%.

[0172] As mentioned above, the timing for a mobile phone to determine whether there are processor resources available to support the operation of the camera requested by the application can be divided into timing 1 and timing 2. Timing 1: During the OpenCamera phase of the application, based on the resource usage information of the logical camera predicted during the initialization phase, it is determined whether there are processor resources available to support the operation of the camera requested by the application. Timing 2: During the application's stream allocation phase, based on the actually selected SensorSettings, it is determined whether there are processor resources available to support the operation of the camera requested by the application.

[0173] In some embodiments, timing 1 and timing 2 can both use the following determination method 1 or determination method 2 to determine whether there are processor resources available to support the operation of the camera requested by the application.

[0174] Judgment Method 1

[0175] In judgment method 1, the terminal can determine the current comprehensive resource usage information based on the resource usage information corresponding to the logical camera, and compare the current comprehensive resource usage information with the upper limit value to obtain the judgment result. For example, if the current comprehensive resource usage information does not exceed the upper limit value, result 1 is obtained; if the current comprehensive resource usage information exceeds the upper limit value, result 2 is obtained. The upper limit value represents the upper limit of processor resources that the terminal can provide; for example, the upper limit value corresponds to the total number of ISP processors that can be provided.

[0176] The resource usage information corresponding to the logical camera can be the resource usage information predicted for the logical camera during the initialization phase, or the resource usage information predicted for the logical camera calculated based on the actually selected Sensor Settings during the application's streaming phase.

[0177] The current overall resource usage information refers to the sum of the resource usage information corresponding to the logical camera that application A is requesting to access and the resource usage information corresponding to the logical camera currently being accessed on the terminal. The logical camera currently being accessed on the terminal is a logical camera that has been successfully accessed and is still being accessed; for example, it could be a logical camera accessed by other applications. As mentioned above, the resource usage information corresponding to a logical camera can also be referred to as the resource usage information corresponding to the application currently calling that logical camera. Therefore, the current overall resource usage information can also be understood as the sum of the resource usage information corresponding to application A and the resource usage information corresponding to other applications on the terminal that have successfully accessed cameras.

[0178] Taking resource usage information as an example, suppose the camera access request is initiated by the system camera to request access to the rear camera corresponding to logical camera 2 for taking a picture. Before the system camera initiates this camera access request, a third-party application has already successfully accessed the front camera corresponding to logical camera 1 to achieve a video call. Therefore, the current comprehensive resource usage value is the sum of the resource usage value 1 corresponding to logical camera 1 used for the video call and the resource usage value 2 corresponding to logical camera 2 that the system camera wants to access. It should be understood that if the resource usage value is the ratio of the number of ISP processors required when a logical camera is accessed by an application to the total number of ISP processors in the terminal, then the upper limit can be 100%. If the resource usage value is the number of ISP processors required when a logical camera is accessed by an application, then the upper limit can be the total number of ISP processors in the terminal.

[0179] Judgment Method 2

[0180] In judgment method 2, the terminal can also obtain currently available resource information (i.e., hardware resource information, such as processor resource information), compare the resource occupancy information corresponding to the logical camera with the currently available resource information, and determine whether the current hardware resources support multiple applications accessing different cameras in parallel, or whether there are currently processor resources that support the operation of the camera (logical camera or physical camera) requested by application A, thus obtaining a judgment result. For example, if the resource occupancy information corresponding to the logical camera is less than the currently available resource information, result 1 is obtained; if the resource occupancy information corresponding to the logical camera is greater than the currently available resource information, result 2 is obtained.

[0181] In some embodiments, combined with Figure 8 The software structure shown includes timing diagrams for some solutions; please refer to the documentation for details. Figure 9 .

[0182] It includes (1.1) initialization phase, (1.2) OpenCamera phase of APP1, (1.3) Configure_stream phase of APP1, (1.4) OpenCamera phase of APP2, and (1.5) Configure_stream phase of APP2.

[0183] (1.1) Initialization phase.

[0184] For example, (1.1) the initialization phase process may include S111 to S112.

[0185] S111: Camera HAL predicts the resource usage value for each logical camera based on the maximum number of ISP processors used by each logical camera.

[0186] For the sake of brevity, Figure 9 The detailed steps of S111 are not illustrated in the text. In fact, in some embodiments, step S111 may include:

[0187] S1111: Camera HAL obtains the sensor drive setting parameters for each physical camera from the sensor driver.

[0188] Specifically, Camera HAL can traverse the sensor drivers of each physical camera and obtain the sensor driver setting parameters of each physical camera.

[0189] S1112: For each logical camera, Camera HAL determines the number of ISP processors used for each set of driver settings parameters included in that logical camera.

[0190] Specifically, Camera HAL determines the number of ISP processors required for each set of driver settings parameters based on the driver settings parameters of the physical camera corresponding to each logical camera.

[0191] S1113: Camera HAL predicts the resource usage value for each logical camera based on the ratio of the maximum number of ISP processors occupied by each logical camera to the total number of ISP processors.

[0192] The maximum number of ISP processors used is the maximum number of ISP processors used among the various sets of driver settings parameters for the logical camera.

[0193] S112: The Framework layer obtains and stores the resource usage value corresponding to each logical camera from the Camera HAL.

[0194] After the initialization phase, each application with camera access can initiate a camera access request to access the camera for image capture. The following illustration uses the example of APP1 initiating camera access first, followed by APP2 initiating camera access in parallel during APP1's access process.

[0195] The processing phase of APP1 initiating camera access can be simply referred to as APP1 accessing the camera. The entire process of APP1 accessing the camera includes at least (1.2) APP1's OpenCamera phase and (1.3) APP1's Configure_stream phase.

[0196] (1.2) The OpenCamera phase of APP1.

[0197] In some embodiments, (1.2) the processing of the OpenCamera phase of APP1 may include S121 to S122.

[0198] S121: APP1 initiates a camera access request to request access to the logic Camera1.

[0199] Among them, Logical Camera1 is the logical camera that APP1 is currently trying to connect to and access in the camera access request.

[0200] S122: The Framework layer confirms and records the predicted resource usage value for logical Camera1.

[0201] It should be noted that in this embodiment, it is assumed that APP1 is the application accessing the logical camera for the first time. That is, before APP1 initiates the camera access request, no resource consumption value has been generated due to camera access. Therefore, the currently available ISP processors are very likely sufficient for APP1's camera access request. Therefore, the comprehensive resource consumption value is not calculated for the camera access request initiated by APP1, nor is it determined whether the comprehensive resource consumption value exceeds the upper limit. Instead, it can directly enter the Configure_stream stage of APP1 in (1.3).

[0202] (1.3) The Configure_stream phase of APP1.

[0203] In some embodiments, (1.3) the processing of the Configure_stream stage of APP1 may include S131 to S134.

[0204] S131: APP1 requests configuration of camera image stream from the Framework layer.

[0205] S132: The Framework layer sends distribution information to the Camera HAL.

[0206] S133: Camera HAL selects a suitable set of driver setting parameters for logic Camera1 based on the distribution information.

[0207] S134: Camera HAL sends driver setting parameters to the camera's sensor driver to power on the physical camera corresponding to logical Camera1.

[0208] Among them, the driver setting parameters sent by Camera HAL to the camera's sensor driver are a set of driver setting parameters actually selected in S133.

[0209] After the distribution is completed, steps S135 to S136 can be executed to acquire the corresponding images for APP1.

[0210] S135:APP1 instructs the Framework layer to start the physical camera to acquire image streams.

[0211] S136: The Framework layer instructs the Camera HAL to initiate the physical camera to acquire an image stream.

[0212] It should be understood that the Camera HAL can control the sensor drive to instruct the physical camera to acquire an image stream and report the acquired image stream layer by layer to APP1 (not shown in the figure).

[0213] During the process of APP1 accessing the camera, APP2 can also initiate camera access requests in parallel to access other physical cameras. The processing stage of APP2 initiating camera access can be simply referred to as APP2 accessing the camera. The entire process of APP2 accessing the camera includes at least (1.4) APP2's OpenCamera stage and (1.5) APP2's Configure_stream stage.

[0214] (1.4) The OpenCamera phase of APP2.

[0215] In some embodiments, (1.4) the processing of the OpenCamera phase of APP2 may include S141 to S145.

[0216] S141: APP2 initiates a camera access request to request access to the logic Camera0.

[0217] S142: The Framework layer determines whether the current total resource usage exceeds the upper limit.

[0218] In S142, the current total resource usage value refers to the sum of the resource usage value predicted during the initialization phase for the currently attempted logical Camera0 and the currently generated resource usage value. The total resource usage value can also be called the sum of resource usage values.

[0219] In some embodiments, the Framework layer can first determine whether the logical Camera0 that is currently being tried to connect is currently occupied. If it is not occupied, then S142 is executed.

[0220] Similarly, if S142 determines that the upper limit value has not been exceeded, proceed to the Configure_stream stage of (1.5)APP2, that is, execute S151 to S154. If S142 determines that the upper limit value has been exceeded, execute S143.

[0221] S143: When the current total resource usage exceeds the upper limit, the Framework layer issues a camera eviction command to APP1.

[0222] S144: Camera HAL can clear the camera stream and instruct the sensor driver to close the physical camera corresponding to the logical Camera1 that APP1 requested to access.

[0223] S145: APP1 responded to an error callback, and the camera screen froze.

[0224] (1.5) The Configure_stream phase of APP2.

[0225] In some embodiments, (1.5) the processing of the Configure_stream stage of APP2 may include S151 to S154.

[0226] S151: APP2 requests configuration of camera image stream from the Framework layer.

[0227] S152: The Framework layer sends distribution information to the Camera HAL.

[0228] S153: Camera HAL selects a suitable set of driver setting parameters for logic Camera0 based on the distribution information.

[0229] S154: The Camera HAL sends driver setting parameters to the camera's sensor driver to power on the physical camera corresponding to the logical Camera0.

[0230] After the distribution is completed, steps S155 to S156 can be executed to acquire the corresponding images for APP2.

[0231] S155:APP2 instructs the Framework layer to start the physical Camera to acquire image streams.

[0232] S156: The Framework layer instructs the Camera HAL to initiate the physical camera to acquire an image stream.

[0233] The Camera HAL can control the sensor drive to instruct the physical camera to acquire an image stream, and then report the acquired image stream layer by layer to APP2 (not shown in the figure).

[0234] In some embodiments, combined with Figure 8 The software architecture shown provides timing diagrams for other solutions; please refer to [link / reference]. Figure 10 .

[0235] It includes (2.1) initialization phase, (2.2) OpenCamera phase of APP1, (2.3) Configure_stream phase of APP1, (2.4) OpenCamera phase of APP2, and (2.5) Configure_stream phase of APP2.

[0236] (2.1) Initialization phase.

[0237] For example, (2.1) the initialization phase process may include S211: Camera HAL determines the number of ISP processors occupied by each group of driver setting parameters included in each logical camera.

[0238] To be concise, Figure 10 The detailed steps of S211 are not illustrated in the text. In fact, in some embodiments, S211 may include S2111 to S2112. S2111 to S2112 can be referred to as S1111 to S1112 described above, and will not be repeated here.

[0239] (2.2) The OpenCamera phase of APP1.

[0240] In some embodiments, (2.2) the processing of the OpenCamera phase of APP1 may include S221 to S222.

[0241] S221: APP1 initiates a camera access request to request access to the logic Camera1.

[0242] S222: The Framework layer establishes a camera access path corresponding to the logical Camera1.

[0243] It should be noted that in this embodiment, APP1 is also used as an example of an application accessing the logical camera for the first time. Please refer to the explanation of S122 above for details.

[0244] After establishing the camera access path, you can enter the Configure_stream stage of APP1 in (2.3).

[0245] (2.3) The Configure_stream phase of APP1.

[0246] In some embodiments, (2.3) the processing of the Configure_stream stage of APP1 may include S231 to S235, wherein S231 to S233 can refer to the above S131 to S133, and will not be repeated here.

[0247] S234: The Camera HAL calculates the number of ISP processors used based on the actual set of driver settings parameters selected, calculates the resource usage value corresponding to the logical Camera1 based on the number of ISP processors used, and returns the resource usage value corresponding to the logical Camera1 to the Framework layer.

[0248] In some embodiments, the Camera HAL can look up the ISP processor usage corresponding to the actually selected set of driver settings parameters based on the ISP processor usage corresponding to each set of driver settings parameters calculated during the initialization phase. In other embodiments, the Camera HAL can also recalculate the ISP processor usage based on the actually selected set of driver settings parameters during the Configure_stream phase of APP1.

[0249] S235: The Camera HAL sends driver setting parameters to the camera's sensor driver to power on the physical camera corresponding to the logical Camera1.

[0250] After the distribution is completed, steps S236 to S237 can be executed to acquire the corresponding images for APP1. Steps S236 to S237 can be referenced from steps S135 to S136 above, and will not be repeated here.

[0251] During the process of APP1 accessing the camera, APP2 can also initiate camera access requests in parallel to access other physical cameras. The processing stage of APP2 initiating camera access can be simply referred to as APP2 accessing the camera. The entire process of APP2 accessing the camera includes at least (2.4) APP2's OpenCamera stage and (2.5) APP2's Configure_stream stage.

[0252] (2.4) The OpenCamera phase of APP2.

[0253] In some embodiments, (2.4) the processing of the OpenCamera phase of APP2 may include S241 to S242.

[0254] S241: APP2 initiates a camera access request to request access to logic Camera0.

[0255] S242: Framework layer logic determines whether Camera0 is currently occupied.

[0256] If it is not occupied, then notify APP2 to enter the Configure_stream stage of APP2 (2.5), such as executing step S251. If it is occupied, then the access logic Camera0 cannot be connected.

[0257] (2.5) The Configure_stream phase of APP2.

[0258] In some embodiments, (2.5) the processing of the Configure_stream stage of APP2 can include S251 to S259, wherein S251-S253 can refer to the above S151-S153.

[0259] S254: The Camera HAL calculates the number of ISP processors used based on the actual set of driver settings parameters selected, calculates the resource usage value corresponding to the logical Camera0 based on the number of ISP processors used, and returns the resource usage value corresponding to the logical Camera0 to the Framework layer.

[0260] In some embodiments, Camera HAL can look up the ISP processor usage corresponding to the actually selected set of driver settings parameters based on the ISP processor usage corresponding to each set of driver settings parameters calculated during the initialization phase. In other embodiments, Camera HAL can also recalculate the ISP processor usage based on the actually selected set of driver settings parameters during the Configure_stream phase of APP2.

[0261] S255: The Framework layer determines whether the current total resource usage exceeds the upper limit.

[0262] If S255 determines that the upper limit value has not been exceeded, then S259 is executed. If S255 determines that the upper limit value has been exceeded, then S256-S258 are executed.

[0263] S256: When the overall resource usage value exceeds the upper limit, the Framework layer issues a camera eviction command to APP1.

[0264] S257: Camera HAL can clear the camera stream and instruct the sensor driver to close the physical camera corresponding to the logical Camera1 that APP1 requested to access.

[0265] S258: APP1 responded to an error callback, and the camera screen froze.

[0266] S259: When the overall resource usage value does not exceed the upper limit, the Camera HAL sends out driver setting parameters to power on the physical camera corresponding to the logical Camera0.

[0267] After the distribution is completed, steps S260 to S261 can be executed to acquire the corresponding images for APP2. Steps S260 to S261 can be referenced from S155 to S156 above, and will not be repeated here.

[0268] It should be understood that the above embodiments all describe releasing processor resources by camera eviction. In some embodiments, the terminal can also reduce the processor resource consumption of applications by lowering the parameter values ​​of specified settings in the corresponding actual Sensor Settings, thereby enabling multiple applications to access different cameras in parallel. Here, the actual Sensor Settings corresponding to the application refers to the Sensor Settings actually selected for the logical camera requested by the application.

[0269] Specifically, the terminal can respond to user actions by exiting the second application and returning to the first interface. If the terminal receives an operation from the user to open a fourth application based on the first interface, the terminal can respond by opening the fourth application and opening the second camera. During the opening of the second camera, the image captured in real time by the first camera is displayed in a floating window of the first application.

[0270] In the Configure_stream stage corresponding to the fourth application, the fourth application is stream-configured. During the stream configuration process for the fourth application, the terminal can obtain the third driver setting parameters selected for the second camera. The third driver setting parameters are automatically selected to meet the image acquisition requirements of the fourth application using the second camera; that is, the images acquired by the second camera using the third driver setting parameters can fully meet the image acquisition requirements of the fourth application.

[0271] In the Configure_stream stage corresponding to the fourth application, the terminal can adjust the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameters; the first setting item in the fourth driver setting parameters corresponds to the second parameter value; the second parameter value is less than the first parameter value. It should be understood that by lowering the first parameter value corresponding to the first setting item in the third driver setting parameters, processor resources can be freed up for the second camera to operate. Furthermore, the terminal can issue the fourth driver setting parameters to the second camera; the fourth driver setting parameters are used to control the second camera to acquire images.

[0272] The first setting is the specified setting.

[0273] After allocating the streaming to the fourth application, a fourth interface is displayed. This solution releases processor resources by lowering the parameter values ​​of some settings in the driver settings during the Configure_stream phase corresponding to the fourth application. Therefore, it does not require camera eviction from other applications, such as the first application. That is, even if the fourth application successfully accesses the second camera, the first application can still access the first camera. Therefore, the fourth interface includes a floating window for the first application and the real-time image captured by the second camera. Within the fourth interface, the floating window for the first application displays the real-time image captured by the first camera.

[0274] In the above scheme, during the traffic allocation stage corresponding to the fourth application, the terminal library reduces the processor resource requirements of the second camera by lowering the parameter values ​​of the settings in the driver settings, thereby enabling the first and fourth applications to access different cameras in parallel.

[0275] In some embodiments, during the process of allocating traffic to the fourth application, adjusting the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameters includes: during the process of allocating traffic to the fourth application, determining whether there are processor resources that support the operation of the second camera based on the third driver setting parameters selected for the second camera; if there are no processor resources that support the operation of the second camera, adjusting the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameters. It should be understood that if it is determined that there are processor resources that support the operation of the second camera, then it is not necessary to adjust the third driver setting parameters; instead, the third driver setting parameters can be directly issued to the second camera.

[0276] In the above scheme, during the traffic allocation stage corresponding to the fourth application, the terminal can accurately determine whether there are processor resources available to support the operation of the second camera based on the third driver setting parameters actually selected for the second camera. Furthermore, if result 2 is obtained (i.e., it is determined that there are currently no processor resources available to support the operation of the second camera), instead of using a camera banning method, the terminal reduces the processor resource requirements of the second camera by lowering the value of the first parameter corresponding to the first setting item in the third driver setting parameters. This allows the first and fourth applications to access different cameras in parallel. In addition, relying on accurate judgment results, processor resources can be utilized more rationally when the first and fourth applications can access different cameras in parallel.

[0277] In some embodiments, the first setting can be the frame rate. For example, after determining that the current overall resource usage exceeds the upper limit, the phone can reduce the frame rate value in the Sensor Settings corresponding to the system camera, thereby reducing the system camera's usage on the ISP processor. This avoids excluding the system camera or other applications, ensuring that multiple applications can access different physical cameras normally in parallel in more scenarios. It is important to emphasize that reducing the frame rate to decrease the system camera's ISP processor usage minimizes the impact on user experience, as a certain degree of frame rate reduction has little effect on the user's visual perception. Moreover, reducing the frame rate does not affect the quality of individual images, achieving the effect of reducing ISP processor resource usage while maintaining image quality.

[0278] It should be understood that the terminal can adjust the parameter value of the first setting item for any application, without limitation. In some embodiments, the terminal can adjust the parameter value of the first setting item for a preset application, wherein the preset application can be the system camera, or it can be other system-provided applications that can access the system camera, or in other words, a third-party application that has been granted permission to lower the permissions of the phone. The first setting item can also be resolution, or it can be both frame rate and resolution. That is, the terminal can adjust the parameter value of the first setting item for a preset application if the fourth application is one of them.

[0279] In some embodiments, the mobile phone can lower the parameter value of the first setting item using either of the following two adjustment methods. The following description uses frame rate as an example to illustrate the two adjustment methods. It should be understood that the processing principle for other first settings items is the same as that for frame rate, and therefore will not be listed individually.

[0280] Adjustment method 1: Adjust according to the preset ratio.

[0281] Taking the frame rate as the first setting as an example, after application A initiates a camera access request, if the judgment result is result 2, that is, the current comprehensive resource usage information exceeds the preset upper limit, the mobile phone can reduce the frame rate of the system camera by 0.5 times according to the preset ratio, such as 0.5, thereby reducing the resource usage value of the system camera and achieving the purpose of releasing ISP processor resources.

[0282] Adjustment method 2: Adjust downwards based on the difference from the upper limit.

[0283] Taking the frame rate as the first setting as an example, after application A initiates a camera access request, if the judgment result is result 2, the mobile phone can calculate the difference between the current comprehensive resource usage information and the upper limit value, and adjust the frame rate of the system camera according to the difference value, so that the reduced comprehensive resource usage information is less than or equal to the upper limit value, thereby releasing ISP processor resources, so that the mobile phone can respond to the camera access request initiated by application A and perform image acquisition.

[0284] To facilitate understanding, let's take the example where the first setting is frame rate and the preset application is the system camera. We will now combine... Figure 11 A more detailed explanation of Scheme 2 will be provided.

[0285] In some embodiments, after the parameter value of the first setting item (i.e., the first setting item) in the corresponding actual Sensor Settings is lowered for an application (such as the fourth application), the mobile phone can generate a prompt message and display the prompt message to notify the user that the parameter value of the first setting item of the preset application has been lowered. For example, if the frame rate of the system camera is lowered, the user can be prompted that the camera's consumption of processor resources exceeds the system's capacity limit, resulting in a lower frame rate.

[0286] It should be understood that the method of adjusting the parameter value of the first setting item in other embodiments of this application can refer to the above adjustment method 1 or adjustment method 2, and will not be described again.

[0287] like Figure 11 As shown, it includes (3.1) initialization phase, (3.2) APP1's OpenCamera phase, (3.3) APP1's Configure_stream phase, (3.4) system camera's OpenCamera phase, and (3.5) system camera's Configure_stream phase.

[0288] (3.1) Initialization phase.

[0289] For example, (3.1) the initialization phase process may include S311, wherein S311 may refer to S211 above, and will not be repeated here.

[0290] To be concise, Figure 11 The detailed steps of S311 are not illustrated in the text. In fact, in some embodiments, S311 may include S3111 to S3112. S3111 to S3112 can be referred to as S1111 to S1112 described above, and will not be repeated here.

[0291] (3.2) The OpenCamera phase of APP1.

[0292] In some embodiments, (3.2) the processing of the OpenCamera stage of APP1 may include S321 to S322. Wherein, S321 to S322 can refer to S221 to S222 described above, and will not be repeated here.

[0293] After establishing the camera access path in the OpenCamera phase of APP1 in (3.2), you can proceed to the Configure_stream phase of APP1 in (3.3).

[0294] (3.3) The Configure_stream phase of APP1.

[0295] In some embodiments, (3.3) the processing of the Configure_stream stage of APP1 may include S331 to S335. Wherein, S331 to S335 may refer to S231 to S235 described above.

[0296] It should be understood that in the Configure_stream stage of APP1 in (3.3), the Framework layer can record the resource usage value calculated based on the actual selected driver settings parameters, which is convenient for subsequent judgment on whether the overall resource usage value exceeds the upper limit.

[0297] After the distribution is completed, steps S336 to S337 can be executed to acquire the corresponding images for APP1. Steps S336 to S337 can be referred to as S135 to S136 above, and will not be repeated here.

[0298] During the process of APP1 accessing the camera, APP2 can also initiate camera access requests in parallel to access other physical cameras. The processing stage of APP2 initiating camera access can be simply referred to as APP2 accessing the camera. The entire process of APP2 accessing the camera includes at least (3.4) APP2's OpenCamera stage and (3.5) APP2's Configure_stream stage.

[0299] (3.4) The OpenCamera phase of APP2.

[0300] In some embodiments, (3.4) the processing of the OpenCamera stage of APP2 may include S341 to S342. Wherein, S341 to S342 may refer to S241 to S242.

[0301] (3.5) The Configure_stream phase of APP2.

[0302] In some embodiments, the processing of the Configure_stream stage of APP2 (3.5) can be included in S351 to S359. Among them, S351 to S355 can refer to S251-S255 above.

[0303] If S355 determines that the upper limit value has not been exceeded, proceed to S360. If S355 determines that the upper limit value has been exceeded, proceed to S356 through S359.

[0304] S356: When the overall resource usage exceeds the upper limit, the Framework layer instructs the Camera HAL to reduce the system camera's resource usage.

[0305] S357: Camera HAL reduces the system camera's workload on the ISP processor by lowering the frame rate corresponding to the system camera.

[0306] S358: The Framework layer updates the resource usage values ​​corresponding to each application.

[0307] Specifically, in step S358, for each application currently accessing the phone's camera, the phone updates the resource usage value of the logical camera corresponding to each application. It should be understood that the resource usage value corresponding to the logical camera requested by the application also belongs to the resource usage value corresponding to that application.

[0308] S359: System camera response callback, generating a notification message indicating a decrease in frame rate.

[0309] S360: When the overall resource usage value does not exceed the upper limit, the Camera HAL sends out driver setting parameters to power on the physical camera corresponding to the logical Camera0.

[0310] After the distribution is completed, steps S361 to S362 can be executed to acquire the corresponding images for APP2. Steps S361 to S362 can be referred to as S155 to S156 or S260 to S261 above, and will not be repeated here.

[0311] In some embodiments, the mobile phone can obtain resource usage information corresponding to the second application during the process of opening the second camera, and determine whether there are processor resources available to support the operation of the second camera based on the resource usage information; wherein, the resource usage information is used to characterize the processor usage of the second camera, which is opened by the second application, during operation. If there are currently no processor resources available to support the operation of the second camera (i.e., in result 2), the mobile phone can adjust the third parameter value corresponding to the first setting item in the fifth driver setting parameters selected for the second camera to obtain a sixth driver setting parameter; the first setting item in the sixth driver setting parameter corresponds to a fourth parameter value; the fourth parameter value is less than the third parameter value; the sixth driver setting parameter is issued to the second camera; the sixth driver setting parameter is used to control the second camera to acquire images.

[0312] It should be understood that the method for adjusting the third parameter value corresponding to the first setting item in the fifth drive setting parameters selected for the second camera can be found in adjustment method 1 or adjustment method 2 described above, and will not be repeated here.

[0313] In the above scheme, the mobile phone can also obtain the resource usage information of the application (such as the OpenCamera stage of the second application) to determine whether there are processor resources available to support the operation of the second camera. If it is determined that there are no processor resources available to support the operation of the second camera, the mobile phone can reduce the processor resource requirements of the second camera by lowering the parameter value of the first setting item in the actually selected driver settings during the traffic allocation stage of the application (such as the second application). This allows multiple applications to access different cameras in parallel, for example, enabling the first application and the second application to access different cameras in parallel.

[0314] In some embodiments, during the process of opening the second camera, such as in the OpenCamera phase of the second application, the resource usage information obtained by the terminal is the first resource usage information. The first resource usage information is the resource usage information predicted based on the target driver setting parameters corresponding to the second camera during the terminal's initialization phase. The target driver setting parameters are the set of driver setting parameters that consumes the most processor resources among the multiple sets of driver setting parameters corresponding to the second camera.

[0315] In the above scheme, during the terminal initialization phase, the terminal predicts the first resource usage information corresponding to the second camera when it is called by the second application based on the set of driver settings parameters corresponding to the second camera that consumes the most processor resources. During the OpenCamera phase of the application, the terminal can determine whether there are processor resources available to support the operation of the second camera based on the first resource usage information predicted during the initialization phase. Therefore, it can handle the situation accordingly based on the current processor resource usage. For example, if it is determined that there are no processor resources available to support the operation of the second camera, during the traffic allocation phase of the second application, the parameter value corresponding to the first setting item in the actually selected driver settings can be lowered, allowing multiple applications to access different cameras in parallel.

[0316] In some embodiments, a mobile phone can set a whitelist. The whitelist includes an application identifier and the resource usage information corresponding to that application identifier. For ease of description, an application whose application identifier is in the whitelist can be called a whitelisted application, and an application whose application identifier is not in the whitelist can be called a non-whitelisted application.

[0317] For example, whitelisted applications may include the system camera or third-party applications used for video calls. Preset use cases may include photo taking, video recording, and video call scenarios.

[0318] It should be noted that the resource usage information recorded in the whitelist is based on empirical values. Compared to the resource usage information predicted based on the set of driver settings parameters corresponding to the camera that consumes the most processor resources, the resource usage information recorded in the whitelist is closer to the actual resource usage information when an application (such as a second application) accesses the camera.

[0319] Therefore, for whitelisted applications, it's unnecessary to predict the resource usage information of the logical camera corresponding to that whitelisted application during the initialization phase. During the process of opening the second camera, such as in the OpenCamera phase of the second application, if the second application is whitelisted, the phone can directly obtain the resource usage information corresponding to that whitelisted application—that is, the second resource usage information. This second resource usage information is the resource usage information recorded for the second application in the whitelist. Furthermore, during the process of opening the second camera, the second resource usage information recorded in the whitelist allows for a more accurate determination of whether there are processor resources available to support the second camera's operation. In addition, directly obtaining the recorded resource usage information from the whitelist reduces computational load, making it more efficient and thus improving camera access efficiency.

[0320] In some embodiments, if processor resources sufficient to support the operation of the second camera are available (i.e., result 1), the image captured in real time by the first camera is displayed in a floating window of the first application during the process of opening the second camera. That is, if result 1 is obtained based on the resource usage information recorded for the application in the whitelist, the application is not banned from using the camera.

[0321] In some embodiments, if there are currently no processor resources available to support the operation of the second camera (i.e., result 2), during the process of opening the second camera, the display of the image captured by the first camera in the floating window of the first application is stopped, or a still image is displayed in the floating window of the first application. That is, if result 2 is obtained based on the resource usage information recorded for the application in the whitelist, the mobile phone can perform camera banning during the OpenCamera stage of the application.

[0322] It should be noted that, based on the resource usage information recorded for applications in the whitelist, if result 2 is obtained, the phone can also reduce the processor resource requirements of the second camera during the data allocation phase for the application (such as the second application). This can be achieved by lowering the parameter value of the first setting item in the actually selected driver settings, thereby enabling multiple applications to access different cameras in parallel. See the description above for details.

[0323] In some embodiments, if the application is a whitelisted application, such as when the application's application identifier is in the whitelist, it can be determined whether there are processor resources available to support the operation of the camera requested by the application based on the resource usage information recorded for the application in the whitelist.

[0324] In other embodiments, the whitelist records resource usage information corresponding to the application in a preset usage scenario. This resource usage information is used to characterize the processor resources required for the application to run the camera accessed in the preset usage scenario. Therefore, during the process of opening the second camera, if the current usage scenario of the second application is the preset usage scenario, the whitelist is used to determine whether there are processor resources available to support the operation of the second camera.

[0325] Specifically, during the process of activating the second camera, the terminal can detect the current usage scenario of the second application. If the second application is a whitelisted application and its current usage scenario is a preset scenario, the terminal can determine whether there are processor resources available to support the operation of the second camera based on the second resource usage information recorded for the second application in the whitelist. It should be understood that the embodiments of this application do not limit the order of the two processes: detecting the current usage scenario of the second application and determining whether the second application is a whitelisted application.

[0326] In some embodiments, if an application is not a whitelisted application, or if an application is a whitelisted application but its current usage scenario does not conform to the preset usage scenario, it can be treated as a non-whitelisted application.

[0327] For non-whitelisted applications, the resource usage information of the corresponding logical camera can still be predicted during the initialization phase. That is, as shown in the initialization phase (1.1) above, for non-whitelisted applications, the phone can predict the resource usage information of the logical camera based on the set of SensorSettings containing the maximum resource usage of the logical camera corresponding to that non-whitelisted application. Therefore, during the OpenCamera phase for non-whitelisted applications, the phone can directly obtain the resource usage information predicted during the initialization phase for the logical camera corresponding to that non-whitelisted application.

[0328] To facilitate understanding, an example is provided below. For instance, when application A (e.g., the second application) initiates a camera access request, during the OpenCamera phase corresponding to application A, the system can detect the current usage scenario and determine if application A is in the whitelist. If application A is in the whitelist, and the current usage scenario matches the preset usage scenario corresponding to application A in the whitelist, then the system retrieves the resource usage information (such as the second resource usage information) corresponding to application A under the preset usage scenario from the whitelist, and uses this as the resource usage information for the logical camera that application A wants to access in this camera access request.

[0329] If application A is not in the whitelist, the resource usage information predicted by the logical camera corresponding to application A during the initialization phase can be obtained.

[0330] Furthermore, during the OpenCamera phase corresponding to application A, the mobile phone can determine whether there are currently processor resources available to support the operation of the camera requested by application A, based on the resource usage information of the camera requested by application A. For specific determination methods, please refer to determination method 1 or determination method 2 above, which will not be repeated here.

[0331] In the above solution, since the whitelisted application uses more realistic resource usage information, during the OpenCamera phase of the whitelisted application, it can determine whether there are enough processor resources available based on this more realistic resource usage information. This allows the whitelisted application to access the camera first, reducing the problem of camera screen freezing or interruption in scenarios where multiple applications access different cameras in parallel, thus improving the user experience.

[0332] Furthermore, the above solution directly stores the resource consumption values ​​of whitelisted applications in preset usage scenarios in the whitelist, reducing computational processing during camera access and improving camera access efficiency. Moreover, the solution retains the predictive processing of resource consumption values ​​during the initialization phase and the processor resource processing logic in the application's OpenCamera phase for determining whether there are currently any cameras that support application A's request for access. This is equivalent to optimizing the solution by incorporating the whitelist while preserving as much of the original processing code as possible, reducing significant code modifications and saving development costs to some extent.

[0333] In some embodiments, if the second application is a whitelisted application (i.e., the second application belongs to the whitelist), if it is determined during the process of opening the second camera that there are currently no processor resources to support the operation of the second camera, the terminal can stop displaying the image captured by the first camera in the floating window of the first application, or display a still image in the floating window of the first application. That is, it stops the first application's access to the first camera, releases processor resources, and thus allocates bandwidth and performs subsequent processing for the second application. In this scheme, applications on the whitelist are given priority during resource allocation to ensure stable access to camera resources.

[0334] This application also provides a chip system applied to a terminal. The chip system includes one or more processors, which are used to invoke computer instructions to cause the terminal to perform the various functions or steps described in the above method embodiments.

[0335] This application also provides a computer-readable storage medium including computer instructions that, when executed on the terminal, cause the terminal to perform the functions or steps described in the method embodiments.

[0336] This application also provides a computer program product that, when run on a computer, causes the computer to perform the functions or steps described in the above method embodiments.

[0337] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0338] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0339] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0340] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0341] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0342] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to a terminal, the method includes: The first interface is displayed, which includes a floating window of the first application, and the floating window of the first application includes images captured in real time by the first camera. In response to opening the second application, the second camera is opened, and during the process of opening the second camera, the image captured in real time by the first camera is displayed in the floating window of the first application; Distribute traffic to the second application; After the second application is allocated the data, a second interface is displayed. The second interface includes a floating window of the first application and images captured in real time by the second camera. In the second interface, the floating window of the first application includes images captured in real time by the first camera.

2. The method according to claim 1, characterized in that, The process of allocating traffic to the second application includes: Based on the first driver setting parameters selected for the second camera, determine whether there are processor resources that support the operation of the second camera. If processor resources that support the operation of the second camera are available, the first driver setting parameters are issued to the second camera; the first driver setting parameters are used to control the second camera to acquire images.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Exit the second application and display the first interface; In response to opening a third application, the second camera is opened, and during the process of opening the second camera, the image captured in real time by the first camera is displayed in the floating window of the first application; The third application is streamed; during the stream distribution process, the image captured by the first camera is stopped from being displayed in the floating window of the first application, or a still image is displayed in the floating window of the first application during the stream distribution process. After the third application is allocated the bandwidth, a third interface is displayed. The third interface includes a floating window of the first application and an image captured in real time by the second camera. In the third interface, the floating window of the first application does not include the image captured by the first camera, or the floating window of the first application displays the still image.

4. The method according to claim 3, characterized in that, The step of stopping the display of the image captured by the first camera in the floating window of the first application during the distribution of the third application, or displaying a still image in the floating window of the first application during the distribution of the third application, includes: During the process of allocating traffic to the third application, the system determines whether there are processor resources that support the operation of the second camera based on the second driver setting parameters selected for the second camera. If there are currently no processor resources available to support the operation of the second camera, stop the first application from accessing the first camera.

5. The method according to claim 1 or 2, characterized in that, The method further includes: Exit the second application and display the first interface; In response to opening the fourth application, the second camera is opened, and during the process of opening the second camera, the image captured in real time by the first camera is displayed in the floating window of the first application; The fourth application is configured with a streaming method; during the streaming of the fourth application, the first parameter value corresponding to the first setting item in the third driving setting parameters selected for the second camera is adjusted to obtain the fourth driving setting parameters; the first setting item in the fourth driving setting parameters corresponds to a second parameter value; the second parameter value is less than the first parameter value; the fourth driving setting parameters are sent to the second camera; the fourth driving setting parameters are used to control the second camera to acquire images; After the fourth application is allocated the bandwidth, a fourth interface is displayed. The fourth interface includes a floating window of the first application and images captured in real time by the second camera. In the fourth interface, the floating window of the first application includes images captured in real time by the first camera.

6. The method according to claim 5, characterized in that, In the process of allocating traffic to the fourth application, the first parameter value corresponding to the first setting item in the third driving setting parameters selected for the second camera is adjusted to obtain the fourth driving setting parameters, including: During the process of allocating traffic to the fourth application, the system determines whether there are processor resources that support the operation of the second camera based on the third driver setting parameters selected for the second camera. If there are currently no processor resources available to support the operation of the second camera, adjust the first parameter value corresponding to the first setting item in the third driver setting parameters selected for the second camera to obtain the fourth driver setting parameter.

7. The method according to claim 5 or 6, characterized in that, The first setting item includes the frame rate; the parameter value of the first setting item includes the frame rate value.

8. The method according to claim 1, characterized in that, The method further includes: During the process of opening the second camera, resource usage information corresponding to the second application is obtained, and the existence of processor resources that support the operation of the second camera is determined based on the resource usage information; wherein, the resource usage information is used to characterize the processor usage of the second camera opened by the second application during operation; In the absence of processor resources to support the operation of the second camera, the flow allocation for the second application includes: Adjust the third parameter value corresponding to the first setting item in the fifth driving setting parameters selected for the second camera to obtain the sixth driving setting parameters; the fourth parameter value corresponds to the first setting item in the sixth driving setting parameters; the fourth parameter value is less than the third parameter value. The sixth driving setting parameter is sent to the second camera; the sixth driving setting parameter is used to control the second camera to acquire images.

9. The method according to claim 8, characterized in that, The resource occupancy information is either the first resource occupancy information or the second resource occupancy information; The first resource usage information is the resource usage information predicted based on the target driver setting parameters corresponding to the second camera during the initialization phase of the terminal; the target driver setting parameters are the set of driver setting parameters that consumes the most processor resources among the multiple sets of driver setting parameters corresponding to the second camera. The second resource usage information is the resource usage information recorded for the second application in the whitelist.

10. The method according to claim 1, characterized in that, The second application is an application recorded in the whitelist; the whitelist records the second resource usage information corresponding to the second application. During the process of opening the second camera, the image captured in real time by the first camera is displayed in the floating window of the first application, including: During the process of opening the second camera, the system determines whether there are processor resources that support the operation of the second camera based on the second resource usage information in the whitelist. If there are processor resources available to support the operation of the second camera, the image captured in real time by the first camera will be displayed in the floating window of the first application during the process of opening the second camera.

11. The method according to claim 10, characterized in that, During the process of opening the second camera, determining whether there are processor resources available to support the operation of the second camera based on the second resource usage information in the whitelist includes: During the process of opening the second camera, if the current usage scenario of the second application is a preset usage scenario, the system determines whether there are processor resources that support the operation of the second camera based on the second resource occupancy information in the whitelist.

12. The method according to claim 10 or 11, characterized in that, The method further includes: If there are currently no processor resources available to support the operation of the second camera, during the process of opening the second camera, stop displaying the image captured by the first camera in the floating window of the first application, or display a still image in the floating window of the first application.

13. A terminal, characterized in that, The terminal includes at least: a plurality of cameras, a memory, and one or more processors; the plurality of cameras are used to acquire images, the memory is used to store computer instructions, and when the one or more processors execute the computer instructions, the terminal performs the method as described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on a terminal, cause the terminal to perform the method as described in any one of claims 1-12.

15. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-12.

Images