Cross-device camera cooperation method and electronic device

Abstract

According to the cross-device camera cooperation method and the electronic device, a first device and a second device are in a multi-screen cooperation state, a screen display interface of the first device is a first user interface, and a second user interface displayed in a first display area in the first user interface is a screen display picture of the second device. The first device responds to the first operation, the second user interface starts to display a shooting window, and the shooting window displays picture content collected through the first camera; if n times of switching operations are received accumulatively at the second moment, the first equipment responds to the n times of switching operations, and the camera for collecting the picture content is switched from the first camera to the second camera; wherein the nth switching operation in the n switching operations is used for indicating to switch the camera for collecting the picture content to the second camera; n is an integer greater than 1. According to the embodiment of the invention, the camera switching stability and the user experience are improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a method for cross-device camera collaboration and an electronic device. Background Art

[0002] Multi-screen collaboration technology is a distributed technology that enables cross-device collaboration. Resources can be shared and collaborative operations can be achieved between different devices. For example, after connecting two electronic devices (such as a mobile phone and a tablet computer), the tablet computer can display the desktop interface of the mobile phone, and the user can operate on the desktop interface of the mobile phone displayed on the tablet computer to make the mobile phone perform corresponding functions. For example, in response to an operation by the user on the desktop interface of the mobile phone displayed on the tablet computer to trigger the video call function of the communication software, the mobile phone can initiate a video call.

[0003] In the scenario of multi-screen collaboration, the process of cross-device camera invocation by an electronic device requires a large amount of computing resources and processing resources, resulting in poor stability and user experience when the electronic device responds to user operations. Summary of the Invention

[0004] Embodiments of this application provide a method for cross-device camera collaboration and an electronic device, which are used to improve the stability of cross-device camera switching and user experience.

[0005] In a first aspect, embodiments of this application provide a method for cross-device camera collaboration, which is applied to a first device. The first device and a second device are in a multi-screen collaboration state. The screen display interface of the first device is a first user interface, and the second user interface displayed in a first display area in the first user interface is the screen display picture of the second device. The method includes: at a first moment, in response to a first operation by the first device, a shooting window starts to be displayed in the second user interface, and the shooting window displays the picture content collected by a first camera; if n switching operations are cumulatively received at a second moment, in response to the n switching operations, the first device switches the camera that collects the picture content from a second camera to a third camera; where the nth switching operation in the n switching operations is used to indicate switching the camera that collects the picture content to the third camera; and n is an integer greater than 1.

[0006] Among them, the first moment is before the second moment. When the second camera and the third camera are different cameras, switching processing is required; when the second camera and the third camera are the same camera, no switching processing needs to be performed, that is, the same camera is used before and after switching; the n switching operations are the switching operations cumulatively received from after the first moment to before the second moment.

[0007] In an embodiment of the present application, through n switching operations, the electronic device can skip the intermediate switching process. During several cumulative switches, it can directly switch according to the switching target of the last operation, shortening the processing duration of the switch, improving the switching efficiency, reducing the task volume and thus reducing the occupation of computing resources, so as to avoid the situation of lag or black screen caused by cross-device camera switching, and improving the stability of the electronic device and the user experience.

[0008] In a possible implementation manner, after the second moment, the method further includes: if 1 switching operation is cumulatively received at the third moment, the first device, in response to the 1 switching operation, switches the camera for collecting the picture content from the second camera to the third camera; the 1 switching operation is used to indicate switching the camera for collecting the picture content to the third camera, and the second camera and the third camera are different cameras. In this way, when the cumulative switching number is 1, the electronic device can also directly process it to ensure the timeliness of task processing.

[0009] In a possible implementation manner, if m switching operations are cumulatively received from after the second moment to before the fourth moment, the method further includes: at the fourth moment, the first device, in response to the m switching operations, switches the camera for collecting the picture content from the second camera to the fourth camera; the m switching operations are used to indicate switching the camera for collecting the picture content to the fourth camera, the second camera and the fourth camera are different cameras, and m is an integer greater than 1. In this way, during a wave of cumulative task processing, there may be other cumulative task processing needs to be processed. According to the same processing method, the task is simplified, and the switch is directly performed with the last cumulative switching operation as the target, reducing the number of intermediate processing times and improving the switching stability and the user experience.

[0010] In a possible implementation manner, if k switching operations are cumulatively received from after the third moment to before the fifth moment, the method further includes: at the fifth moment, the first device, in response to the k switching operations, switches the camera for collecting the picture content from the third camera to the fifth camera; the k switching operations are used to indicate switching the camera for collecting the picture content to the fifth camera, the second camera and the fifth camera are different cameras, and k is an integer greater than 1. In this way, after the task processing of 1 operation, there may be other cumulative task processing needs to be processed. According to the same processing method, the task is simplified, and the switch is directly performed with the last cumulative switching operation as the target, reducing the number of intermediate processing times and improving the switching stability and the user experience.

[0011] In a possible implementation, the first camera is a local camera of the first device; the second camera is a local camera of the first device or a camera of the second device. In this way, when the electronic device starts up, the camera is defaulted to its own camera, and subsequently, the user can adjust the camera based on needs, ensuring the feasibility of cross-device switching.

[0012] In a possible implementation, the method further includes: when receiving the i-th switching operation, the first device creates an i-th camera closing task and an i-th camera starting task based on the i-th switching operation; and adds the i-th camera closing task and the i-th camera starting task to a task queue; the i-th switching operation is any one of the n switching operations, and i is an integer from 1 to n in sequence; the first device processes the first camera closing task and the n-th camera starting task in the task queue. In this way, based on the user operation, the electronic device can create and add corresponding tasks, ensuring the reliability of the switching process.

[0013] In a possible implementation, the first device processes the first camera closing task and the n-th camera starting task in the task queue, specifically including: when the task in the first task list is processed, the first device copies the tasks in the task queue to a second task list and clears the tasks in the task queue; processes the tasks in the second task list that meet the preset conditions, and does not process the tasks that do not meet the preset conditions; where the tasks that meet the preset conditions include the first camera closing task and the n-th camera starting task. In this way, after the first task list (old task list) is processed, the tasks in the task queue can be copied to generate a new task list (second task list). So as to continue processing the new task list, ensuring the feasibility of the processing.

[0014] In a possible implementation, the preset condition is that the task time in the second task list is the earliest or the latest. In this way, the preset condition can screen out the operations that need to be processed, improving the accuracy of the switching.

[0015] In a possible implementation, the first device includes a task manager, a task processing module, and a session management module. The first device creates an i-th camera shutdown task and an i-th camera startup task based on the i-th switching operation, specifically including: the task manager creates the i-th camera shutdown task and the i-th camera startup task based on the i-th switching operation; adding the i-th camera shutdown task and the i-th camera startup task to the task queue, specifically including: the task processing module adds the i-th camera shutdown task and the i-th camera startup task to the task queue; the first device processes the first camera shutdown task and the n-th camera startup task in the task queue, specifically including: when all tasks in the first task list are processed, the session management module copies all tasks in the task queue to the second task list and performs a task screening and processing process on the second task list; the second task list after screening only includes the first camera shutdown task and the n-th camera startup task; the first device turns off the first camera in response to the first camera shutdown task; and turns on the second camera in response to the n-th camera startup task. In this way, specific modules can implement corresponding task contents based on their respective functions, ensuring the rationality and efficiency of task processing in the task queue.

[0016] In a possible implementation, the process of task screening and processing is as follows: obtain a first task according to the traversal order of the second task list, determine whether the first task meets the preset condition, and process the first task if the first task meets the preset condition; do not process the first task if the first task does not meet the preset condition; determine that the first task has been processed; and end the process of task screening and processing when all tasks in the second task list have been processed. In this way, through the specific screening and processing process, it is possible to determine whether to process each task response, ensuring the necessity of processing, deleting unnecessary processing processes, and ensuring the stability and efficiency of processing.

[0017] In a possible implementation, the process of task screening and processing is as follows: add the tasks in the second task list that meet the preset condition to the third task list, and process the tasks in the third task list in sequence; discard the tasks in the second task list that do not meet the preset condition. In this way, through the specific screening and processing process, it is possible to determine whether to process each task response, ensuring the necessity of processing, deleting unnecessary processing processes, and ensuring the stability and efficiency of processing.

[0018] Second aspect, an embodiment of the present application provides a cross-device camera collaboration method. A cross-device camera collaboration method is characterized in that it is applied to a first device. The first device and a second device are in a multi-screen collaboration state. The screen display interface of the first device is a first user interface. A second user interface displayed in a first display area of the first user interface is the screen display picture of the second device. The second user interface displays a shooting window, and the shooting window displays the picture content collected by a first camera. The method includes: if n switching operations are cumulatively received at a first moment, the first device responds to the n switching operations and switches the camera for collecting the picture content from the first camera to a second camera; where the nth switching operation in the n switching operations is used to indicate switching the camera for collecting the picture content to the second camera; n is an integer greater than 1.

[0019] In an embodiment of the present application, through n switching operations, the electronic device can skip the intermediate switching process. Among the cumulative several switchings, it can directly switch according to the switching target of the last operation, shortening the processing duration of the switching, improving the switching efficiency, reducing the task volume and thus reducing the occupation of computing resources, so as to avoid the situation of jamming or black screen caused by cross-device camera switching, and improving the stability of the electronic device and the user experience.

[0020] In a possible implementation manner, the method further includes: if 1 switching operation is cumulatively received at a second moment, the first device responds to the 1 switching operation and switches the camera for collecting the picture content from the second camera to a third camera; the 1 switching operation is used to indicate switching the camera for collecting the picture content to the third camera, and the second camera and the third camera are different cameras. In this way, when the cumulative switching number is 1, the electronic device can also directly process it to ensure the timeliness of task processing.

[0021] In a possible implementation manner, if m switching operations are cumulatively received from after the second moment to before a third moment, the method further includes: at a fourth moment, the first device responds to the m switching operations and switches the camera for collecting the picture content from the second camera to a fourth camera; the m switching operations are used to indicate switching the camera for collecting the picture content to the fourth camera, the second camera and the fourth camera are different cameras, and m is an integer greater than 1. In this way, during a wave of cumulative task processing, there may be other cumulative task processing needs to be processed. According to the same processing method, the task is simplified, and it is directly switched according to the last cumulative switching operation as the target, reducing the intermediate processing times and improving the switching stability and the user experience.

[0022] In a possible implementation manner, the first camera is a local camera of the first device; the second camera is a local camera of the first device or a camera of the second device. In this way, when the electronic device just starts up, the camera is defaulted to its own camera, and subsequently, the user can adjust the camera based on needs to ensure the feasibility of cross-device switching.

[0023] In a possible implementation manner, the method further includes: when receiving the i-th switching operation, the first device creates an i-th camera closing task and an i-th camera starting task based on the i-th switching operation; and adds the i-th camera closing task and the i-th camera starting task to the task queue; the i-th switching operation is any one of the n switching operations, and i is an integer from 1 to n in sequence; the first device processes the first camera closing task and the n-th camera starting task in the task queue. In this way, based on the user operation, the electronic device can create and add corresponding tasks to ensure the reliability of the switching process.

[0024] In a possible implementation manner, when the first device processes the first camera closing task and the n-th camera starting task in the task queue, it specifically includes: when the task in the first task list is processed, the first device copies the tasks in the task queue to the second task list and clears the tasks in the task queue; processes the tasks in the second task list that meet the preset conditions, and does not process the tasks that do not meet the preset conditions; where the tasks that meet the preset conditions include the first camera closing task and the n-th camera starting task. In this way, after the first task list (old task list) is processed, the tasks in the task queue can be copied to generate a new task list (second task list). So as to continue to process the new task list and ensure the feasibility of the processing.

[0025] In a possible implementation manner, the preset condition is that the task time in the second task list is the earliest or the latest. In this way, the preset condition can screen out the operations that need to be processed to improve the accuracy of the switching.

[0026] In a possible implementation, the first device includes a task manager, a task processing module, and a session management module. The first device creates an i-th camera shutdown task and an i-th camera startup task based on the i-th switching operation, which specifically includes: the task manager creates the i-th camera shutdown task and the i-th camera startup task based on the i-th switching operation; adding the i-th camera shutdown task and the i-th camera startup task to the task queue specifically includes: the task processing module adds the i-th camera shutdown task and the i-th camera startup task to the task queue; the first device processes the first camera shutdown task and the n-th camera startup task in the task queue, which specifically includes: when all tasks in the first task list are processed, the session management module copies all tasks in the task queue to the second task list and performs a task screening and processing process on the second task list; the second task list after screening only includes the first camera shutdown task and the n-th camera startup task; the first device turns off the first camera in response to the first camera shutdown task; and turns on the second camera in response to the n-th camera startup task. In this way, specific modules can implement corresponding task contents based on their respective functions, ensuring the rationality and efficiency of task processing in the task queue.

[0027] In a possible implementation, the process of task screening and processing is as follows: obtain a first task according to the traversal order of the second task list, determine whether the first task meets the preset condition, and if the first task meets the preset condition, process the first task; if the first task does not meet the preset condition, do not process the first task; determine that the first task has been processed; and if all tasks in the second task list have been processed, end the process of task screening and processing. In this way, through a specific screening and processing process, it is possible to determine whether to process each task response, ensuring the necessity of processing, deleting unnecessary processing processes, and ensuring the stability and efficiency of processing.

[0028] In a possible implementation, the process of task screening and processing is as follows: add the tasks in the second task list that meet the preset condition to the third task list, and process the tasks in the third task list in sequence; discard the tasks in the second task list that do not meet the preset condition. In this way, through a specific screening and processing process, it is possible to determine whether to process each task response, ensuring the necessity of processing, deleting unnecessary processing processes, and ensuring the stability and efficiency of processing.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors and one or more memories; the one or more processors are coupled to the one or more memories, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the first device and the second device are in a multi-screen collaboration state, the screen display interface of the first device is the first user interface, and the second user interface displayed in the first display area of the first user interface is the screen display picture of the second device, so that the first device executes:

[0030] The method includes: at a first moment, in response to a first operation, a shooting window starts to be displayed in the second user interface, and the shooting window displays the picture content collected by a first camera; if n switching operations are cumulatively received at a second moment, in response to the n switching operations, the camera for collecting the picture content is switched from a second camera to a third camera; where the nth switching operation in the n switching operations is used to indicate switching the camera for collecting the picture content to the third camera; n is an integer greater than 1.

[0031] Wherein, the first moment is before the second moment. When the second camera and the third camera are different cameras, a switching process is required; when the second camera and the third camera are the same camera, no switching process needs to be performed, that is, the same camera is used before and after the switching; the n switching operations are the switching operations cumulatively received from after the first moment to before the second moment.

[0032] In the embodiment of the present application, through n switching operations, the electronic device can skip the intermediate switching process. In the cumulative several switching operations, it can directly switch according to the switching target of the last operation, shortening the processing duration of the switching, improving the switching efficiency, reducing the task volume and thus reducing the occupation of computing resources, so as to avoid the situation of freezing or black screen caused by cross-device camera switching, and improving the stability and user experience of the electronic device.

[0033] In a possible implementation manner, after the second moment, the first device further executes: if 1 switching operation is cumulatively received at a third moment, in response to the 1 switching operation, the camera for collecting the picture content is switched from the second camera to the third camera; the 1 switching operation is used to indicate switching the camera for collecting the picture content to the third camera, and the second camera and the third camera are different cameras. In this way, when the cumulative number of switching operations is 1, the electronic device can also directly process it to ensure the timeliness of task processing.

[0034] In a possible implementation, if m switching operations are cumulatively received after the second moment and before the fourth moment, the first device further performs: at the fourth moment, in response to the m switching operations, switching the camera for collecting the picture content from the second camera to the fourth camera; the m switching operations are used to indicate switching the camera for collecting the picture content to the fourth camera, the second camera and the fourth camera are different cameras, and m is an integer greater than 1. In this way, during a wave of cumulative task processing, there may be other cumulative task processing that needs to be processed. According to the same processing method, the task is simplified, and the switching is directly performed with the last switching operation of the cumulative as the target, reducing the number of intermediate processing times and improving the switching stability and user experience.

[0035] In a possible implementation, if k switching operations are cumulatively received after the third moment and before the fifth moment, the first device further performs: at the fifth moment, in response to the k switching operations, switching the camera for collecting the picture content from the third camera to the fifth camera; the k switching operations are used to indicate switching the camera for collecting the picture content to the fifth camera, the third camera and the fifth camera are different cameras, and k is an integer greater than 1. In this way, after the task processing of 1 operation, there may be other cumulative task processing that needs to be processed. According to the same processing method, the task is simplified, and the switching is directly performed with the last switching operation of the cumulative as the target, reducing the number of intermediate processing times and improving the switching stability and user experience.

[0036] In a possible implementation, the first camera is the local camera of the first device; the second camera is the local camera of the first device or the camera of the second device. In this way, when the electronic device just starts, the camera defaults to its own camera, and the user can adjust the camera based on needs later, ensuring the feasibility of cross-device switching.

[0037] In a possible implementation, the first device further performs: when receiving the i-th switching operation, creating an i-th camera closing task and an i-th camera starting task based on the i-th switching operation; and adding the i-th camera closing task and the i-th camera starting task to the task queue; the i-th switching operation is any one of the n switching operations, and i is an integer from 1 to n in sequence; processing the first camera closing task and the n-th camera starting task in the task queue. In this way, based on the user operation, the electronic device can create and add corresponding tasks to ensure the reliability of the switching process.

[0038] In a possible implementation, the first device processes the first camera shutdown task and the nth camera startup task in the task queue, which specifically includes: when the task in the first task list is completed, copying the tasks in the task queue to the second task list and clearing the tasks in the task queue; processing the tasks in the second task list that meet the preset conditions, and not processing the tasks that do not meet the preset conditions; where the tasks that meet the preset conditions include the first camera shutdown task and the nth camera startup task. In this way, after the first task list (old task list) is processed, the tasks in the task queue can be copied to generate a new task list (second task list). So as to continue to process the new task list and ensure the feasibility of the processing.

[0039] In a possible implementation, the preset condition is that the task time in the second task list is the earliest or the latest. In this way, the preset condition can screen out the operations that need to be processed, improving the accuracy of the switch.

[0040] In a possible implementation, the first device includes a task manager, a task processing module, and a session management module. The first device creates the first camera shutdown task and the first camera startup task based on the i-th switching operation, which specifically includes: the task manager creates the first camera shutdown task and the first camera startup task based on the i-th switching operation; adding the first camera shutdown task and the first camera startup task to the task queue specifically includes: the task processing module adds the first camera shutdown task and the first camera startup task to the task queue; the first device processes the first camera shutdown task and the nth camera startup task in the task queue, which specifically includes: when all the tasks in the first task list are processed, the session management module copies all the tasks in the task queue to the second task list and performs a task screening and processing process on the second task list; only the first camera shutdown task and the nth camera startup task are included in the second task list after screening; in response to the first camera shutdown task, the first camera is turned off; in response to the nth camera startup task, the second camera is turned on. In this way, the specific module can implement the corresponding task content based on the corresponding function, ensuring the rationality and efficiency of processing the tasks in the task queue.

[0041] In a possible implementation, the process of the task screening process is as follows: obtain the first task according to the traversal order of the second task list, determine whether the first task meets the preset conditions, and process the first task when the first task meets the preset conditions; when the first task does not meet the preset conditions, do not process the first task; determine that the first task has been processed; when all tasks in the second task list have been processed, end the process of the task screening process. In this way, through the specific screening process, it is possible to determine whether to process each task response, ensure the necessity of processing, delete unnecessary processing processes, and ensure the stability and efficiency of processing.

[0042] In a possible implementation, the process of the task screening process is as follows: add the tasks in the second task list that meet the preset conditions to the third task list, and process the tasks in the third task list in sequence; discard the tasks in the second task list that do not meet the preset conditions. In this way, through the specific screening process, it is possible to determine whether to process each task response, ensure the necessity of processing, delete unnecessary processing processes, and ensure the stability and efficiency of processing.

[0043] In a fourth aspect, an embodiment of the present application provides an electronic device, including: one or more processors and one or more memories; the one or more processors are coupled to the one or more memories, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the cross-device camera collaboration method according to any one of the possible implementation manners in the second aspect.

[0044] In a fifth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processors are used to call computer instructions to make the electronic device execute the cross-device camera collaboration method according to any one of the possible implementation manners in the first aspect or the second aspect.

[0045] In a sixth aspect, an embodiment of the present application provides a computer program product containing instructions. When the computer program product runs on an electronic device, the electronic device is made to execute the cross-device camera collaboration method according to any one of the possible implementation manners in the first aspect or the second aspect.

[0046] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions. When the instructions run on an electronic device, the electronic device is made to execute the cross-device camera collaboration method according to any one of the possible implementation manners in the first aspect or the second aspect. Brief Description of the Drawings

[0047] Figures 1A to 1C FIG. is a schematic diagram of a user interface for multi-screen collaboration provided by an embodiment of the present application;

[0048] Figures 2A to 2C FIG. is a schematic diagram of a user interface for multi-screen collaboration provided by an embodiment of the present application;

[0049] Figure 3 FIG. is a schematic diagram of a user interface when a mobile phone and a tablet are in collaborative connection provided by an embodiment of the present application;

[0050] Figure 4 and Figure 5 FIG. is a schematic diagram of a user interface for device collaboration provided by an embodiment of the present application;

[0051] Figure 6 FIG. is a schematic diagram of a camera of a collaborative device provided by an embodiment of the present application;

[0052] Figure 7 FIG. is a schematic diagram of an interface for camera switching provided by an embodiment of the present application;

[0053] Figure 8 FIG. is another schematic diagram of an interface for camera switching provided by an embodiment of the present application;

[0054] Figure 9 FIG. is a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application;

[0055] Figure 10 FIG. is a schematic diagram of a process flow of a cross-device camera collaboration method between two devices provided by an embodiment of the present application;

[0056] Figure 11 FIG. is a schematic diagram of a user interface for high-frequency cross-device camera switching provided by an embodiment of the present application;

[0057] Figure 12 FIG. is another schematic diagram of a user interface for high-frequency cross-device camera switching provided by an embodiment of the present application;

[0058] Figure 13 FIG. is a schematic diagram of a camera switching process provided by an embodiment of the present application;

[0059] Figure 14 FIG. is a schematic diagram of a process flow of a method for camera switching in cross-device collaboration provided by an embodiment of the present application;

[0060] Figure 15 FIG. is another schematic diagram of a camera switching process provided by an embodiment of the present application;

[0061] Figure 16It is a schematic diagram of the software structure of another electronic device provided by an embodiment of the present application;

[0062] Figures 17A to 17C It is a schematic diagram of the method flow for camera switching in cross-device collaboration provided by an embodiment of the present application;

[0063] Figure 18 It is a schematic diagram of the method flow for task screening and processing provided by an embodiment of the present application;

[0064] Figures 19A to 19D It is a schematic diagram of the task processing process provided by an embodiment of the present application;

[0065] Figure 20 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0066] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are also intended to include the plural forms, unless clearly indicated to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0067] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more than two. In addition, in order to clearly describe the technical solutions in the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit to be different.

[0068] During a video conference, the user needs to record relevant information content. While the mobile phone is in video, the computer also needs to be convenient to operate. In view of the above scenario, the need for cross-device collaboration is proposed. In order to meet the different needs of users to use devices such as mobile phones and computers, mobile phones and tablets, and tablets and computers across devices, mobile phones, tablets, laptops, etc. have the function of cross-device collaboration. After multi-screen collaboration connects devices such as mobile phones, computers, and tablets in a coordinated manner, resource sharing, coordinated operation, etc. can be achieved.

[0069] To facilitate understanding, the following uses the collaboration between a mobile phone and a tablet computer as an example to introduce several possible connection methods for multi-screen collaboration.

[0070] 1.Bluetooth connection:

[0071] Combination Figures 1A to 1C The user interface diagram of multi-screen collaboration shown in the figure illustrates the Bluetooth connection process:

[0072] 1) Enable multi-screen collaboration on your tablet.

[0073] like Figure 1A As shown, the user can slide down from the top right side of the tablet screen to open the control center, click to expand the quick switch bar 110, and the quick switch bar 110 can include at least a multi-screen collaboration 111 control and a Bluetooth control. Figure 1A In the shortcut switch bar 110, WIFI control, Bluetooth control, mute control, automatic rotation control, NFC control, multi-screen collaboration 111 control, do not disturb control and screenshot control, etc. The user can click the multi-screen collaboration 111 control (light up or turn on multi-screen collaboration).

[0074] Optionally, the user can enable the multi-screen collaboration function through other paths. For example, the user can find the "multi-device collaboration" switch through the path of "Settings"-"More Connections"-"Multi-Device Collaboration", and set the switch to the on state, so as to enable the multi-screen collaboration function of the tablet. Among them, the method of enabling the multi-screen collaboration of the tablet is not limited in this application.

[0075] 2) Turn on Bluetooth on your phone.

[0076] like Figure 1B As shown, the user can slide down the control center from the top (right side) of the mobile phone to display the quick switch bar 120. The quick switch bar 120 may include a Bluetooth 121 control, and the user can click the Bluetooth 121 control (to light up or turn on Bluetooth).

[0077] Alternatively, the user can turn on Bluetooth through other paths, for example, find the "Bluetooth" switch through the path of "Settings"--"Bluetooth", set the switch to the Bluetooth state, and thus turn on the Bluetooth function of the mobile phone. Among them, the method of turning on the Bluetooth of the mobile phone is not limited in this application.

[0078] 3) Bring your phone close to your tablet and follow the pop-up prompts on the phone and tablet screens to complete the connection.

[0079] like Figure 1C As shown in (a), when the mobile phone detects a device requesting to establish a collaborative connection, the user interface of the mobile phone may display a request pop-up window 130. The request pop-up window 130 is used to prompt the user to request to establish a collaborative connection. Figure 1CIn (a), the request pop-up window 130 includes prompt information: "Do you want to establish a collaborative connection with the discovered device?", a connect 131 control, and a cancel 132 control. The connect 131 control is used to determine the establishment of a collaborative connection; the cancel 132 control is used to not establish a collaborative connection. When the user clicks the connect 131 control, Figure 1C As shown in (b), the user interface of the mobile phone may display a connection pop-up window 140. The connection pop-up window 140 may indicate that a connection is currently being made with a certain device, and the connection pop-up window 140 may include a cancel control so that the user can stop the connection. Optionally, the connection pop-up window 140 may also include information such as the model of the collaboratively connected device (not shown), which is not limited in the present application.

[0080] 2. Scan the QR code to connect:

[0081] Combination Figures 2A to 2C The user interface diagram of multi-screen collaboration shown in the figure illustrates the process of scanning code to connect:

[0082] 1) Enable multi-screen collaboration on your tablet and click Scan QR code to connect on the multi-screen collaboration interface.

[0083] Figure 2A Combination Figure 1A As described above, the user can slide down from the top right side of the tablet screen to display the control center, click to expand the quick switch bar 210, and the user can click the multi-screen collaboration 211 control in the quick switch bar 210 to turn on multi-screen collaboration. After turning on multi-screen collaboration, the tablet can display a prompt message. Figure 2B As shown, the tablet displays a multi-screen collaboration prompt pop-up window 220. The multi-screen collaboration shortcut switch bar 210 may include a description of the collaborative connection method of the relevant devices. The prompt pop-up window 220 may include a "scan code to connect" 221 control. The user clicks the "scan code to connect" 221 control, such as Figure 2C As shown, the electronic device displays a QR code pop-up window 230 , and the QR code pop-up window 230 includes a QR code 231 .

[0084] 2) Complete the connection according to the prompts on the tablet scan code connection interface.

[0085] When the mobile phone is connected to the Internet, it can scan the QR code 231 on the tablet to connect, and follow the prompts to connect. Figure 1C And the related descriptions are not repeated here.

[0086] In one example, on the mobile phone side, the user can enter an interface including the "Scan" option in the browser (or Smart Vision). In response to the user's trigger operation on the "Scan" option, the mobile phone activates the camera, so that the user can aim the camera at the QR code displayed on the tablet side for scanning. After the mobile phone successfully scans the code, it sends a request to establish a connection to the tablet. After receiving the request from the mobile phone, the tablet can display a prompt message, such as "Device xx requests to establish a connection with this end. Do you agree to establish a connection?" In addition, the prompt pop-up window can also include an "Agree" option and a "Disagree" option. In response to the user's trigger operation on the "Agree" option, the tablet establishes a connection with the mobile phone.

[0087] It should be noted that the above is only an example of opening the QR code through the path of "My Phone" - "Connect Immediately" - "Scan Code to Connect" on the tablet side. In another embodiment, the QR code can also be opened through other paths, and the present application is not limited thereto.

[0088] 3. Super Terminal Connection:

[0089] The two devices log in to the same account, search for nearby devices for binding, and perform collaborative connection:

[0090] The mobile phone and the tablet turn on Bluetooth, are in an Internet-connected state, and log in to an account (for example, Honor account). The mobile phone and the tablet need to log in to the same account. If the tablet has not logged in to the terminal account or is not the same account, enter Settings > Super Terminal > This Device > Allow to be Discovered, and please confirm to select nearby devices. Slide down from the upper right side of the mobile phone to bring out the control center, and click in the Super Terminal: Search for Nearby Devices and bind new devices. Click on the corresponding tablet icon in the mobile phone Super Terminal to complete the collaborative connection.

[0091] 4. Tap to Connect:

[0092] The mobile phone enables the NFC function to make a one-touch connection with the NFC area of the tablet keyboard, and completes the connection based on the prompt messages of the mobile phone and the tablet.

[0093] It should be noted that if it is a collaborative connection between the mobile phone and the laptop, the tablet and the laptop can be connected by wire using a data cable, and the connection is made based on the prompt message. It can also be connected by tapping the keyboard: the mobile phone enables NFC to make a one-touch connection with the NFC of the laptop keyboard. In addition, multi-screen collaboration operations can be performed between the tablet and the laptop, between the mobile phone and the tablet, and between the mobile phone and the laptop. The multi-screen collaboration connection methods between different types are diverse, and the present application is not limited thereto.

[0094] After the mobile phone and the tablet are successfully connected, the tablet can mirror the window of the mobile phone. In this way, the user can operate within this window according to needs.

[0095] Figure 3 It is a schematic diagram of the user interface when the mobile phone and the tablet are in cooperative connection.

[0096] When the mobile phone and the tablet are in cooperative connection, the mobile phone window can be mirror-displayed on the tablet, and the user can operate within this window according to needs. For example, Figure 3 As shown, the mobile phone mirror 311 is displayed in the tablet interface 310, that is, the mobile phone interface 320 is cast to the tablet interface 310. At this time, it is more efficient to use mobile phone applications, drag and drop for mutual transfer, copy and paste text, and edit mobile phone files on the tablet.

[0097] Optionally, when the mobile phone and the tablet complete the cooperative connection, the mobile phone and the tablet respectively display a cooperative logo. For example, Figure 3 In the figure, the cooperative-in-progress logo 321 of the mobile phone is displayed in the upper left corner of the mobile phone interface 320, and the cooperative-in-progress logo 312 of the tablet is displayed in the upper left corner of the tablet interface 310, indicating that the current devices are in cooperative connection.

[0098] It should be noted that the above pictures are only examples of cooperation between two devices. It can also be cooperation among 3 or more devices. For example, cooperation between two mobile phones and a tablet, and two mobile phone windows can be mirror-displayed on the tablet; another example is cooperation among a mobile phone, a tablet and a computer, and the mobile phone window and the tablet window can be mirror-displayed on the computer, etc. The present application is not limited to this.

[0099] During the cooperation of multiple devices, the master device can switch to the audio and video of other devices it cooperates with, and multiple cameras of multiple devices can be called. The third-party application software of the device can call the camera of the device. During the cooperation of multiple devices, the camera called by the third-party application software can be its own camera or the camera of the cooperative device, and the user can switch the camera through a switching operation.

[0100] Figure 4 and Figure 5 are a set of schematic diagrams of the user interface of device cooperation publicly exemplified in the embodiments of the present application. As Figure 4 and Figure 5 shown, taking the cooperation between the mobile phone and the tablet as an example, multiple cameras in the cooperation screen can all be used.

[0101] Next, in combination with Figure 4 and Figure 5 , several situations of the application software will be exemplarily described:

[0102] In one example, when a user wants to make a video call through a communication software, the user can click on the icon of communication software A in this window of the tablet to open the communication software, and then click on the "Video Call" option in the communication software. In response to the user's triggering operation on the "Video Call" option, the tablet computer sends a video call control instruction to the mobile phone. After receiving the video call control instruction, the mobile phone initiates a video call request to make a video call with other users. Please refer to Figure 4 , during this process, the screens of the windows of the mobile phone and the tablet synchronously display the video call interface.

[0103] In another example, when a user is live streaming through the mobile phone, the user can click on the icon of live streaming software B in this window of the tablet (or mobile phone) to open the live streaming software, and then click on the "Start Live Streaming" option in the live streaming software. In response to the user's triggering operation on the "Start Live Streaming" option, the tablet sends a live streaming control instruction to the mobile phone. After receiving the live streaming control instruction, the mobile phone initiates a live stream. Please refer to Figure 5 , during this process, the screens of the windows of the mobile phone and the tablet synchronously display the user live streaming interface.

[0104] In yet another example, when a user wants to take a photo, the user can click on the icon of instant camera C in this window of the tablet to take a photo. In response to the user's triggering operation on the icon of camera C, the tablet computer sends a shooting instruction to the mobile phone. After receiving the shooting instruction, the mobile phone initiates a shooting request to take a photo. During this process, the screens of the windows of the mobile phone and the tablet synchronously display the shooting interface.

[0105] In the shooting of the above-mentioned video call, live streaming, camera and other third-party application software, during the multi-device collaboration process, the rapid switching of the shooting positions can be realized. In addition, the switching scenarios between multiple cameras in multi-screen collaboration are diverse. The above three scenarios are only examples. For another example, video conferencing, etc. The present application is not limited to this.

[0106] Figure 6 is a schematic diagram of the camera of a collaborative device provided exemplarily in an embodiment of the present application. As Figure 6 shown in (a) therein, the tablet may include two cameras, namely a front camera and a rear camera. Exemplarily, the IDs corresponding to the front camera and the rear camera of the tablet computer are 0 and 1 respectively. As Figure 6 shown in (b) therein, the mobile phone may include four cameras, namely a front camera, a rear telephoto camera, a rear wide-angle camera and a rear main camera. Exemplarily, the identifiers (identifiers, IDs) corresponding to the front camera, the rear telephoto camera, the rear wide-angle camera and the rear main camera of the mobile phone are 1, 0, 6 and 7 respectively.

[0107] In addition, if the laptop is used as a collaborative device, the laptop may include a camera, which may be a front camera. Exemplarily, the ID of the front camera of the laptop is 0. If the foldable screen mobile phone is used as a collaborative device, the foldable screen mobile phone may include a front camera on the outer screen (the third screen), a rear telephoto camera, a rear wide-angle camera, and a rear main camera. The foldable screen mobile phone may also include a front camera on the inner screen. Exemplarily, the IDs of the front camera on the inner screen, the rear main camera, the rear telephoto camera, the rear wide-angle camera, and the front camera on the outer screen of the foldable screen mobile phone are 1, 0, 9, 8, and 13 respectively.

[0108] In the case where the third-party application calls the camera as described above, the electronic device may provide a function of switching the camera. The following will be described in conjunction with Figure 7 and Figure 8 as follows:

[0109] Figure 7 FIG. is a schematic diagram of an interface for switching the camera exemplarily disclosed in an embodiment of the present application.

[0110] In a possible embodiment, after the mobile phone and the tablet complete the collaborative connection and during the process of the third-party application using the camera, the notification bar of the tablet may display a multi-screen collaboration notification. As Figure 7 shown, when the user slides down from the upper right side of the tablet to pull out the control center, in response to the above operation, the electronic device may also display a quick switch bar 711 and a notification bar. The notification bar may include a multi-screen collaboration notification 712, and the multi-screen collaboration notification 712 may include a prompt message indicating that it has been connected to the tablet, and may also include a "disconnect" control. The user can click the disconnect control to end the collaboration. It may also include a disconnect button and a button for switching the audio and video to the mobile phone. The user can adjust the opening and closing of the corresponding buttons according to needs. It should be understood that when the tablet computer is connected (collaborated) to the mobile phone, the camera of the tablet computer may be used by default for video picture acquisition. If the user only wants to use the camera of the mobile phone itself to collect video pictures, the user can operate the button for switching the audio and video to the mobile phone (for example, click the button for switching the audio and video to the mobile phone). In this way, the video picture during the video call can be collected by the camera of the mobile phone.

[0111] Optionally, the notification bar includes a multi-screen collaboration notification on the mobile phone side. The multi-screen collaboration notification may include a prompt message indicating that it is connected to the mobile phone, and provides "disconnect" and "switch audio and video to mobile phone" controls. The user can click the "disconnect" control to end the collaboration between the mobile phone and the tablet, and the user can click the "switch audio and video to mobile phone" control to switch the picture and voice.

[0112] Figure 8 FIG. is another schematic diagram of an interface for switching the camera exemplarily disclosed in an embodiment of the present application.

[0113] In another possible embodiment, when multi-screen collaboration is established between a mobile phone and a tablet computer, after the user opens the camera application on the mobile phone, the user can select a local camera or the camera of the tablet computer through the path "More" - "Multi-camera". For the mobile phone, the camera of the tablet computer can be regarded as a virtual camera. As Figure 8 shown, in response to the user's operation of triggering the selection of a camera in the camera application of the mobile phone, the mobile phone can display a pop-up window 801, and the pop-up window 801 can include multiple camera options. The multiple camera options can include options corresponding to the local camera and options corresponding to the camera of the tablet computer. Among them, the local camera can include the local front camera (i.e., the device itself (front)), the local rear main camera (i.e., the device itself (rear)), the local wide-angle camera (i.e., the device itself (wide-angle)), and the camera of the tablet computer can include the front camera of the tablet computer (i.e., my tablet (front)), the rear main camera of the tablet computer (i.e., my tablet (rear)), the wide-angle camera of the tablet computer (i.e., my tablet (wide-angle)). Optionally, the local camera can further include a local telephoto camera, a macro camera, etc. The camera of the tablet computer can also include a telephoto camera, a macro camera, etc. of the tablet computer. In this way, in the virtual camera scenario (i.e., the scenario where an electronic device (such as a mobile phone) calls a camera across devices), the mobile phone can call multiple types of cameras (such as a front camera, a rear main camera, a wide-angle camera, etc.) across devices. The mobile phone can provide multiple types of cameras for the user to choose from, which can enrich the user's choices and improve the user experience.

[0114] It should be noted that due to the differences in third-party applications, the ways to open and switch the camera function interface are different, and the display methods of the camera switching interface (such as a pop-up window) are also different. The number and identification of cameras for collaboration between different devices can also be different, and the camera switching interface can also be different. Figure 7 and Figure 8 Both are exemplary processes for camera switching, and the present application does not limit them.

[0115] Figure 9 is a schematic diagram of the software structure exemplarily provided by an embodiment of the present application.

[0116] As Figure 9 shown, the layered architecture of the electronic device involved in the present application divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into an application layer, an application framework layer, a system layer, an extension layer, and a kernel layer from top to bottom.

[0117] The application layer can include a series of application packages. As Figure 8As shown, the application package may include but is not limited to video, navigation, SMS, camera, shared camera, communication software, XX live broadcast, XX conference, multi-screen collaboration and other applications. Among them, the communication software application can be used to realize video calls; the XX live broadcast application can be used to realize video live broadcast, the XX conference application can be used to realize video conferencing, and the multi-screen collaboration application can be used to enable the multi-screen collaboration function. The shared camera application can be used to realize the function of calling the camera across devices.

[0118] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 8 As shown, the application framework layer may include a service discovery process, a collaborative assistant process, and an interconnected service process.

[0119] The service discovery process is used to monitor the connection instruction for instructing collaboration after Bluetooth or NFC is turned on, and notify the collaboration assistant process after monitoring the connection instruction. The collaboration assistant process is used to establish a collaborative connection by exchanging information with the collaboration assistant processes in other electronic devices after receiving the notification from the service discovery process.

[0120] For example, during a video call, if it is in a collaborative scenario, the interconnection service process is used to enable the virtualized camera service to open up a data transmission channel between the interconnection service process and the virtual camera process. In addition, after the virtualized camera service is successfully enabled, the interconnection service process is also used to send a video stream to the calling device (master control device).

[0121] Exemplarily, the interconnection service process includes a data processing module, a transmission channel module, a flow control module and a capability acquisition module. Among them, the data processing module can be used to process the video frame according to the needs of the underlying layer, such as format conversion, etc.; the transmission channel module is used to configure the transmission channel; the flow control module is used to cache the video stream; the capability acquisition module is used to collect the camera capabilities of the local electronic device and send the camera capabilities of the local electronic device to the calling device (master control device).

[0122] The system layer includes the audio framework, multimedia framework, and virtual camera adaptation process.

[0123] The audio framework can support a variety of commonly used audio encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The multimedia framework can support a variety of commonly used video formats and static image files, etc. The video formats supported by the multimedia framework can include, for example, AVI, WMV, FLV, Blu-ray, etc.

[0124] Among them, the video formats supported by the multimedia framework may include, for example: AVI, WMV, FLV, Blu-ray, etc.

[0125] Among them, the virtual camera adaptation process is mainly used to perform format conversion processing on the data transmitted between the interconnected service process and the extension layer. It can be understood that the virtual camera adaptation process is a bridge for data transmission between the interconnected service process and the virtual camera process. Exemplarily, when the interconnected service process needs to transmit data to the virtual camera process, the interconnected service process first sends the data to the virtual camera adaptation process, and the virtual camera adaptation process performs format conversion, that is, converts it into a data format that the virtual camera process can recognize and process, and then sends the converted data to the virtual camera process. Conversely, when the virtual camera process needs to transmit data to the interconnected service process, the virtual camera process first sends the data to the virtual camera adaptation process, and the virtual camera adaptation process performs format conversion on the data, that is, converts it into a data format that the interconnected service process can recognize and process, and then sends the converted data to the interconnected service process. Due to the existence of the virtual camera adaptation process, the virtual camera process does not need to concern itself with the upper-layer logic, and the interconnected service process does not need to care about the characteristics of the underlying chip.

[0126] The virtual camera adaptation process may include a first service, a second service, a converter, and a camera relationship mapping module.

[0127] Among them, the first service is used to provide services to the upper layer and support the first data format. For example, the first service is the CHANNEL service. If the interconnected service process needs to send data to the underlying virtual camera process, it can first convert the data according to the first data format, and then send the converted data to the virtual camera adaptation process through the first service, so that the virtual camera adaptation process sends it to the virtual camera process.

[0128] The second service is used to provide services to the lower layer and support the second data format. For example, the second service is the TRANSLATOR service. If the virtual camera process needs to send data to the upper-layer interconnected service process, it can first convert the data according to the second data format, and then send the converted data to the virtual camera adaptation process through the second service, so that the virtual camera adaptation process sends it to the interconnected service process.

[0129] The converter is used to perform format conversion on the data. For example, convert the underlying data into a data format that the upper layer can recognize and process, or convert the upper-layer data into a data format that the underlying layer can recognize and process.

[0130] Among them, the converter may include a camera mapping module. The camera mapping module is used to map the camera (or camera) of the local electronic device to the camera of the controlled device. For example, in the case where it is determined according to the collected camera capabilities that both the local electronic device and the controlled device include a front camera and a rear camera, the front camera of the local electronic device is corresponded to the front camera of the controlled device, and the rear camera of the local electronic device is corresponded to the rear camera of the controlled device.

[0131] The extended layer may include a camera process and a virtual camera process, and the camera process and the virtual camera process are located in the HAL (hardware abstract layer). Among them, the camera process is used to turn on the local camera according to the service requirements of the application layer and collect video frames through the local camera. In one embodiment, in a collaborative scenario, the camera process is used to send the video frames collected by the local camera to the interconnection service process according to the service requirements of the virtual camera process, so that the interconnection service process can send the video frames collected by the local camera to the calling device (master device).

[0132] The kernel layer is the layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, a sensor driver, etc. Among them, the camera driver is used to drive the camera hardware so that the camera starts, and thus, the electronic device can collect images through the camera.

[0133] Combined with Figure 9 the software architecture, Figure 10 is a schematic flow diagram of a cross-device camera collaboration method between two devices disclosed in an embodiment of the present application.

[0134] Next, taking the establishment of a collaborative connection between the first device and the second device as an example, a cross-device camera collaboration method between the first device and the second device will be introduced in detail. Among them, the first device may be a calling end (master end), and the second device may be a called end (controlled end). At this time, the first device and the second device are in a multi-screen collaboration state, and the screen display interface of the first device is the first user interface, and the second user interface displayed in the first display area of the first user interface is the screen display picture of the second device.

[0135] As Figure 10 shown, the method may include but is not limited to the following steps:

[0136] S1001. In response to the first operation, the first application of the first device sends a request to open the camera to the multi-screen collaboration application.

[0137] Among them, the request to open the camera may carry the identifier (ID) of the local camera of the first device.

[0138] Multi-screen collaboration is established between the first device and the second device. When the user triggers a video call on the first device, the first application of the first device sends a request to open the camera to the multi-screen collaboration application of the first device.

[0139] The first application may be a third-party application, such as communication software, live broadcast software, camera, conference software, etc. The first operation refers to an operation in the first application for starting to use the camera, such as the operation of the user clicking to start a video call in the communication software interface; the operation of turning on the camera; the operation of starting a live broadcast; the operation of turning on the camera in the conference software, etc. The above operations all require starting to capture the picture.

[0140] In some embodiments, after the multi-screen collaboration is established between the mobile phone and the tablet computer, in response to the user triggering a video call on the mobile phone, the communication software of the mobile phone can send a request to the multi-screen collaboration application of the mobile phone to open the camera. The operation of triggering the video call may include initiating a video call or answering a video call, which is not limited in this application.

[0141] In other embodiments, when the mobile phone has initiated a video call, in response to the user triggering an operation to establish multi-screen collaboration between the mobile phone and the tablet computer, the mobile phone's communication software may send a request to open the camera to the multi-screen collaboration application of the mobile phone.

[0142] S1002: The multi-screen collaborative application of the first device sends a request to enable a virtualized camera service to the interconnection service process of the first device.

[0143] The interconnection service process can enable the virtualized camera service to open up a data transmission channel between the interconnection service process and the virtual camera process.

[0144] S1003: The interconnection service process of the first device requests the capability aggregation module of the first device for camera information of the second device.

[0145] S1004: The capability aggregation module of the first device requests camera information from the capability collection module of the second device.

[0146] The capability aggregation module of the first device may send a request message to the capability acquisition module of the second device through its own transmission channel module, where the request message is used to request the camera information of the tablet computer.

[0147] S1005. The capability acquisition module of the second device requests camera information from the multimedia framework of the second device.

[0148] The capability acquisition module of the second device may receive a request message from the first device through its own transmission channel module, where the request message is used to request camera information of the second device.

[0149] S1006. The multimedia framework of the second device obtains the camera information of the second device.

[0150] The camera information of the second device includes the identifiers and camera attributes of all available cameras of the second device.

[0151] Among them, the identifier of the camera can also be called the camera number, which is used to identify different cameras. The identifiers corresponding to different cameras are different. Exemplarily, the IDs corresponding to the front camera, the rear telephoto camera, the rear wide-angle camera, and the rear main camera of a mobile phone are 1, 0, 6, and 7 respectively.

[0152] The camera attributes can include an orientation attribute, and the orientation attribute includes a front (Front) attribute and a rear (Main) attribute.

[0153] Exemplarily, a tablet computer can call the first interface at the system layer to read the orientation attribute of each camera. Among them, the first interface can be, for example, CameraCharacteristics.LENS_FACING. For each camera of the tablet computer, the first interface can be called according to the identifier (ID) corresponding to the camera to read the orientation attribute of the camera. If the return value of the first interface is CameraMetadata.LENS_FACING_FRONT, it means that the camera is a front camera, and the orientation attribute of the camera can be marked as the front (FRONT) attribute; if the return value of the first interface is CameraMetadata.LENS_FACING_BACK, it means that the camera is a rear camera, and the orientation attribute of the camera can be marked as the rear (BACK) attribute.

[0154] In a possible design, the camera attributes can also include a label attribute, and the label attribute includes at least one of a wide-angle attribute, a main camera attribute, a telephoto attribute, or a macro attribute.

[0155] Exemplarily, the tablet computer can call the second interface at the system layer to read the tag attributes. Among them, the second interface can be, for example, KeyGenerator.generateCharacteristicsKey. For each camera of the tablet computer, the second interface can be called according to the identifier (ID) corresponding to the camera to read the tag attributes of the camera. If the return value (key value) of the second interface is com.hihonor.device.capabilities.mainModeSupported, it is considered that the camera is the rear main camera, and the tag attributes of the camera are marked as the main camera (Main) attributes; if the return value (key value) of the second interface is com.hihonor.device.capabilities.teleSupported, it is considered that the camera is a telephoto camera, and the tag attributes of the camera are marked as telephoto attributes; if the return value (key value) of the second interface is com.hihonor.device.capabilities.wideModeSupported, it is considered that the camera is a wide-angle camera, and the tag attributes of the camera are marked as wide-angle attributes.

[0156] Optionally, the camera attributes can also include other attributes, such as magnification attributes, exposure attributes, etc. Among them, the magnification attributes can include, for example, high magnification attributes and low magnification attributes, etc. The exposure attributes can include, for example, high exposure rate attributes and low exposure rate attributes, etc.

[0157] Optionally, the camera information of the tablet computer can also include stream capability information. Among them, the stream capability information can include the bitstreams supported by the camera (for example, H265 / H264, etc.) and the resolutions supported by the camera (for example, 480P / 720P, etc.). Among them, all available cameras of the tablet computer can include the cameras built in the tablet computer and the cameras externally connected to the tablet computer, which are not limited in this application.

[0158] S1007. The multimedia framework of the second device returns the camera information of the second device to the capability acquisition module of the second device.

[0159] S1008. The capability acquisition module of the second device returns the camera information of the second device to the capability aggregation module of the first device.

[0160] The capability acquisition module of the second device can send the camera information of the second device to the first device through its own transmission channel module. The first device can receive the camera information of the second device from the second device through its own transmission channel module and can send the camera information of the second device to the capability aggregation module of the first device.

[0161] The capability aggregation module of the first device stores the camera information of the second device.

[0162] In a possible design, if the camera information from the second device does not include the number of cameras of the second device, the capability aggregation module of the first device can determine the number (count) of cameras of the second device. For example, the capability aggregation module of the first device can count the number of cameras of the second device according to the number of camera identifiers carried in the camera information of the second device. For example, the camera identifiers carried in the camera information of a tablet computer can include "Camera1" and "Camera0", that is, the number of cameras of the tablet computer can be 2.

[0163] S1010. The capability aggregation module of the first device returns the camera information of the second device to the camera mapping module of the first device.

[0164] In some embodiments, after multi-screen collaboration is established between a mobile phone and a tablet computer, in response to a user's operation of triggering a video call on the mobile phone, the communication software of the mobile phone can send a request to open the camera to the multi-screen collaboration application of the mobile phone. Among them, the operation of triggering a video call can include initiating a video call or answering a video call, which is not limited in this application.

[0165] In other embodiments, when the mobile phone has initiated a video call, in response to a user's operation of triggering the establishment of multi-screen collaboration between the mobile phone and the tablet computer, the communication software of the mobile phone can send a request to open the camera to the multi-screen collaboration application of the mobile phone.

[0166] It should be noted that when multi-screen collaboration is established between the first device and the second device, and the user triggers a video call on the first device, on the one hand, steps S1001 - S1010 can be executed to obtain the camera information of the second device. On the other hand, steps S1011 - S1014 can be executed to obtain the camera attributes of the currently opened camera of the first device. Finally, the mapping binding between the local camera (invoking the camera of an electronic device (such as a mobile phone)) and the virtual camera (that is, the camera of the invoked electronic device (such as a tablet computer)) can be performed according to the camera attributes of the currently opened camera of the first device and the camera information of the second device, that is, step S1015 can be executed.

[0167] S1011. The first application of the first device sends a request to the camera HAL of the first device to open the local camera.

[0168] Among them, the request to open the camera can carry the identifier of the local camera. The identifier of the local camera of the first device can be, for example, "Camera 0".

[0169] S1012. The camera HAL of the first device turns on the local camera.

[0170] The camera HAL of the first device can turn on the local camera through the camera driver.

[0171] S1013. The camera HAL of the first device sends the video stream captured by the local camera to the first application of the first device.

[0172] The camera HAL of the first device can obtain the video stream captured by the local camera through the camera driver, and then send the video stream captured by the local camera to the first application of the first device. So that the first application of the first device can display the video image to the user in a timely manner.

[0173] S1014. The camera HAL of the first device sends the orientation attribute and / or marking attribute of the local camera to the camera mapping module.

[0174] In a possible design, the camera HAL of the first device can call the third interface to obtain the orientation attribute of the currently turned-on local camera. Among them, the third interface can be, for example, cameraInfo.facing. The return value of cameraInfo.facing can include CAMERA_FACING_FRONT and CAMERA_FACING_BACK.

[0175] Among them, CAMERA_FACING_FRONT represents the front attribute, and CAMERA_FACING_BACK represents the rear attribute. Taking the currently turned-on local camera as camera 1 as an example, if the orientation attribute corresponding to camera 1 is CAMERA_FACING_FRONT, it is considered that camera 1 is a front camera, and the orientation attribute of camera 1 is marked as Front. If the orientation attribute corresponding to camera 1 is CAMERA_FACING_BACK, it is considered that camera 1 is a rear camera, and the orientation attribute of camera 1 is marked as Main.

[0176] The camera HAL of the first device can call the fourth interface to determine the marking attribute of the currently turned-on local camera. Taking the chip adopted by the first device as the Qualcomm chip 8650 as an example. The fourth interface can include IsMainSensor, IsWideSensor, IsteleSensor, etc. Among them, the return values of interfaces such as IsMainSensor, IsWideSensor, IsteleSensor, etc. can be true or false.

[0177] The camera HAL of the first device can sequentially call interfaces such as IsMainSensor, IsWideSensor, and IsteleSensor to determine the marked attributes of the currently opened local camera. The order in which the mobile phone calls interfaces such as IsMainSensor, IsWideSensor, and IsteleSensor can be a preset order or an order determined according to preset rules, which is not limited in this application.

[0178] S1015. The camera mapping module of the first device performs mapping binding between the local camera and the virtual camera.

[0179] The camera mapping module of the first device can perform mapping binding between the local camera and the virtual camera according to the camera information of the second device.

[0180] In the embodiments of this application, the mapping binding between the local camera and the virtual camera can at least include the following three cases.

[0181] In the first case, when there is only a single camera on the second device, that is, when the number of cameras on the second device is 1, there is a mapping relationship between the local camera and the only 1 camera on the second device, and the camera attributes of the tablet computer do not need to be considered.

[0182] It should be understood that the local camera of the first device can be any local camera of the first device. For example, it can be a front camera, a rear camera, a wide-angle camera, etc., which is not limited in this application. The 1 camera on the second device can be any type of camera. For example, it can be a front camera or a rear camera, which is not limited in this application.

[0183] In the second case, when there are 2 cameras on the second device, the virtual camera that has a mapping relationship with the local camera is the camera among the 2 cameras on the second device that has the same orientation attribute as the local camera. That is, when the number of cameras on the second device is 2, the orientation attributes of the cameras on the second device and the orientation attribute of the local camera can be considered. The local camera has a mapping relationship with the camera on the tablet computer with the same orientation attribute.

[0184] It can be understood that when there are 2 cameras on the second device, the 2 cameras include a front camera and a rear camera.

[0185] Exemplarily, if the orientation attribute of the local camera is the front-facing attribute, that is, the local camera is a front-facing camera, then the camera in the tablet computer with the orientation attribute of the front-facing attribute can be called, that is, the front-facing camera of the tablet computer can be called. If the orientation attribute of the local camera is the rear-facing attribute, that is, the local camera is a rear-facing camera, then the camera in the tablet computer with the orientation attribute of the rear-facing attribute can be called, that is, the rear-facing camera of the tablet computer can be called. In this way, the virtual camera mapped by the local camera of the mobile phone (that is, the camera of the tablet computer) can be determined according to the orientation attribute, and it can be ensured that when the mobile phone calls the camera across devices, the virtual camera with the same orientation attribute as the local camera can be called, which better meets the user's needs.

[0186] In the third case, when there are more than 2 cameras on the second device, the camera that has a mapping relationship with the local camera is the camera among the multiple (more than 2) cameras on the second device that has the same label attribute as the local camera. That is, when the number of cameras on the second device is greater than 2, the marking attributes of the cameras on the second device and the marking attributes of the local camera can be considered. The local camera has a mapping relationship with the camera on the second device with the same marking attribute.

[0187] Exemplarily, if the orientation attribute of the local camera is the wide-angle attribute, that is, the local camera is a wide-angle camera, then the camera in the tablet computer with the orientation attribute of the wide-angle attribute can be called, that is, the wide-angle camera of the tablet computer can be called. If the orientation attribute of the local camera is the telephoto attribute, that is, the local camera is a telephoto camera, then the camera in the tablet computer with the orientation attribute of the telephoto attribute can be called, that is, the telephoto camera of the tablet computer can be called.

[0188] In this way, the virtual camera mapped by the local camera of the mobile phone (that is, the camera of the tablet computer) can be determined according to the marking attribute, and it can be ensured that when the mobile phone calls the camera across devices, the virtual camera with the same marking attribute as the local camera can be called, which better meets the user's needs.

[0189] S1016. The camera mapping module of the first device sends an indication message to the capability aggregation module of the first device.

[0190] Among them, the indication message is used to indicate the camera of the second device that has a mapping relationship with the local camera.

[0191] S1017. The capability aggregation module of the first device sends an indication message to the capability acquisition module of the second device.

[0192] S1018. The capability acquisition module of the second device sends an indication message to the multimedia framework of the second device.

[0193] After receiving the indication information, the multimedia framework of the second device may collect video data (second video data) based on the camera indicated by the indication information.

[0194] S1019. The multimedia framework of the second device sends the video data collected by the camera of the second device to the camera mapping module of the first device.

[0195] The multimedia framework of the second device can send the video data (video frame / video stream) collected by the camera of the second device to the first device through the transmission channel module of the second device. The camera mapping module of the first device can receive the video data sent by the second device through the transmission channel module of the first device.

[0196] S1020. The camera mapping module of the first device sends video data to the camera HAL of the first device.

[0197] S1021. The camera HAL of the first device sends video data collected by the camera of the second device to the first application.

[0198] The communication application of the first device displays a second video window according to the video data collected by the camera of the second device.

[0199] That is, the communication application of the first device can replace the video data collected by the local camera according to the video data collected by the camera of the second device, thereby realizing camera switching.

[0200] Based on the method provided in the present application, in a virtualized camera scenario (i.e., a scenario in which a first device (e.g., a mobile phone) calls the camera of a second device (e.g., a tablet computer) across devices), the first device can consider information such as the number, orientation attributes, and marking attributes of the camera of the second device when calling the camera across devices, so that the mobile phone and the tablet computer can perform camera collaboration more accurately.

[0201] In the above cross-device camera collaboration, if the device switches cameras frequently, the electronic device will be overloaded and there is a risk of freezing or black screen. Figure 11 and Figure 12 Two high-frequency switching scenarios are exemplified:

[0202] In a possible implementation, in cross-device camera collaboration, cameras are switched in a fixed order.

[0203] Figure 11 : is a schematic diagram of a user interface for high-frequency switching of cameras across devices disclosed in an exemplary embodiment of the present application. Figure 11As shown, the options for the mobile phone camera can include six, and these six options are located at different positions in sequence. If the user needs to switch the camera, they can click the "Next" control in the pop-up window. The current camera in the pop-up window 1101 is the local (front) camera. If the user wants to switch to the camera of my tablet (main camera), they need to click the "Next" control four times. In response to four operations (multiple switching operations), the electronic device needs to perform four switches to call the captured image of the camera of my tablet (main camera).

[0204] In another possible implementation, in the cross-device camera collaboration, the user quickly clicks the switching control.

[0205] Figure 12 It is another schematic diagram of the user interface for cross-device high-frequency camera switching disclosed exemplarily in the embodiments of the present application. As Figure 12 shown, the options for the camera can also include six. The user continuously clicks to switch multiple times (multiple switching operations), and clicks the controls corresponding to the local (rear), local (wide-angle), my tablet (front), and my tablet (main camera) in sequence to switch.

[0206] It should be noted that Figure 11 and Figure 12 the camera switching processes are all described exemplarily. The options, quantity, and types of the cameras can all be adaptively changed, and the switching operations are also exemplary processes, all of which are not limited.

[0207] Combined with Figure 11 and Figure 12 the switching processes, Figure 13 it is a schematic diagram of a camera switching process shown in the embodiments of the present application. As Figure 13As shown, at time T0, the third-party application calls the camera 1 to capture the screen content. Before time T0, it can be that the electronic device has just turned on the camera, or it can be that several switching operations have been performed, which is not limited. At times T1, T2, T3, and T4, the user switched 4 times in a high frequency. In response to the user's four switching operations (4 switching operations have been accumulated at time T4, and the corresponding tasks have not been processed), starting from time T5, the electronic device switches the third-party application from camera 1 to camera 2 to capture images. Starting from time T6, the electronic device switches the third-party application from camera 2 to camera 3 to capture images. At time T7, the electronic device switches the third-party application from camera 3 to camera 4 to capture images. From time T8 to time T9, the electronic device switches the third-party application from camera 4 to camera 5 to capture images. After a period of time, after time T9, the third-party application can end the call to the camera and end the current image capture process; it can also continue to ensure the switching of camera 5, and it can also receive new switching operations, which is not limited. Among them, camera 1, camera 2, camera 3, camera 4, and camera 5 can be the local device (front camera), the local device (rear camera), the local device (wide-angle camera), my tablet (front camera), and my tablet (main camera) in sequence. The figure is only an exemplary situation, which is not limited.

[0208] So far, after the user's four switches, the electronic device determines that the target camera for switching is camera 5. The multiple switching processes in the middle are all unnecessary switching processes. The switching occupies storage and computing resources, and also causes too many use tasks to pile up on the electronic device, the processing time is too long, the processing efficiency is reduced, and the stability of the electronic device is poor.

[0209] Figure 14 It is a schematic flowchart of a method for camera switching in cross-device collaboration exemplarily disclosed in this application. As Figure 14 shown, the method for cross-device multi-camera collaboration may include but is not limited to the following steps:

[0210] Similarly, taking the establishment of a collaborative connection between the first device and the second device as an example, the method for cross-device camera collaboration between the first device and the second device will be introduced in detail. Among them, the first device can be the calling end (master control end), and the second device can be the called end (controlled end). That is, the display screen of the second device is mirror-projected onto the screen of the first device. In the embodiments of this application, the first device may include at least camera 1, and the second device may include at least camera 2. Camera N is a camera of the first device. For the first device, the camera of the first device is a local camera, and the camera of the second device that collaborates with it is a virtual camera.

[0211] It should be noted that Figure 14 The method flow shown is Figure 10Steps after the method process in [reference] is executed, that is, the current camera mapping module has established a mapping and binding relationship between the local camera and the virtual camera (S1015). After the user turns on the first camera, the captured image can be obtained through the first camera, and the mirror projection screen in the first device includes the image captured by Camera 1.

[0212] S1401 - S1419 are the processing flows corresponding to a user switching operation. In the embodiments of the present application, a switching operation needs to execute two corresponding tasks, that is, closing the currently running camera and turning on the target camera.

[0213] S1401: In response to the first switching operation, the first application of the first device sends a request to the shared collaboration application to close the first camera.

[0214] During Figure 10 the execution, when the user clicks the first operation, Camera 1 has been turned on. Before executing S1401, there is already a camera capturing images.

[0215] Correspondingly, the shared collaboration application receives the request from the first application to close the first camera. The request to close the first camera is used to request the closing of Camera 1, and the request to close the first camera may include the identifier (ID) of Camera 1. The specific meaning of the camera identifier can be referred to Figure 10 in the relevant description in [reference].

[0216] When the display interface of the first device includes a control for switching cameras, the first switching operation is the operation of the user touching the camera switching control. For details, please refer to Figure 11 and Figure 12 the operations in [reference], which will not be elaborated here.

[0217] S1402: The shared collaboration application of the first device sends a first enabling request to the interconnected service process.

[0218] Correspondingly, the interconnected service process can receive the first enabling request from the shared collaboration application. The first enabling request can indicate enabling the virtual camera service, and the first enabling request may include the identifier of Camera 1, indicating to close Camera 1.

[0219] When the interconnected service process receives the first enabling request, it enables the virtual camera service and determines that it is necessary to close Camera 1. After the interconnected service process receives the first enabling request, the interconnected service process enables the virtual camera service.

[0220] S1403: The interconnected service process of the first device sends a first request to switch to the virtual camera to the first service.

[0221] Correspondingly, the first service can receive a first request to switch to a virtual camera from the interconnected service process. The first request to switch to a virtual camera can include the identifier of Camera 1 to indicate turning off Camera 1.

[0222] S1404: The first service of the first device sends first task information to the camera mapping module.

[0223] After the first service receives the first request to switch to a virtual camera, it can send first task information to the camera mapping module. Correspondingly, the camera mapping module can receive the first task information from the first service. The first task information is used to indicate turning off the camera before the first switch. The first task information includes the identifier of Camera 1.

[0224] Based on the identifier of Camera 1 in the first task information and the mapping relationship, the camera mapping module determines that the camera before the first switch is Camera 1 of the first device. Among them, the specific descriptions of the mapping binding and the mapping relationship can refer to the relevant descriptions in S1015 and S1016, which will not be elaborated here. After the camera mapping module determines the camera before the switch, it can issue a task instruction to turn off the camera before the switch (Camera 1).

[0225] S1405: The camera mapping module of the first device sends a first task instruction to the second service.

[0226] The camera mapping module can send a first task instruction to the second service. The first task instruction is used to indicate turning off Camera 1. Correspondingly, the second service can receive the first task instruction from the camera mapping module.

[0227] S1406: The camera mapping module of the first device sends a first enabling success notification to the interconnected service process through the first service.

[0228] After the camera mapping module sends the first task instruction to the second service, it can send a first enabling success notification to the interconnected service process through the first service. Correspondingly, the interconnected service process can receive the first enabling success notification from the camera mapping module through the first service. Among them, the enabling success notification is used to notify the interconnected service process that the virtual camera service has been enabled.

[0229] S1407: The second service of the first device sends a Camera 1 closing instruction to the camera HAL.

[0230] Correspondingly, the camera HAL of the first device receives the Camera 1 closing instruction from the second service. The Camera 1 closing instruction is used to indicate turning off Camera 1.

[0231] S1408: The camera HAL of the first device turns off Camera 1.

[0232] After the camera HAL of the first device receives the camera 1 closing instruction from the second service, it can close camera 1. Specifically, the camera HAL can call the camera driver to close camera 1 and end the image acquisition process of camera 1.

[0233] S1409: The camera HAL of the first device sends a first closing success notification to the camera mapping module through the second service.

[0234] Correspondingly, the camera mapping module can receive the first closing success notification from the camera HAL through the second service. The closing success notification is used to notify that camera 1 has been successfully closed.

[0235] S1410: In response to the first switching operation, the first application of the first device sends a request to start the second camera to the shared collaboration application.

[0236] Correspondingly, the shared collaboration application of the first device receives the request to start the second camera from the first application. The request to start the second camera includes the identifier of camera 2.

[0237] Among them, the process of S1410 can refer to the relevant description of S1401 and will not be elaborated.

[0238] S1411: The shared collaboration application of the first device sends a second enabling request to the interconnected service process.

[0239] The second enabling request may include the identifier of camera 2 and is used to indicate starting camera 2.

[0240] Among them, the process of S1411 can refer to the relevant description of S1402 and will not be elaborated.

[0241] S1412: The interconnected service process of the first device sends a second request to switch to the virtual camera to the first service.

[0242] The second request to switch to the virtual camera may include the identifier of camera 2.

[0243] Among them, the process of S1412 can refer to the relevant description of S1403 and will not be elaborated.

[0244] S1413: The first service of the first device sends second task information to the camera mapping module.

[0245] After the first service receives the second request to switch to the virtual camera, it can send the second task information to the camera mapping module. The second task information is used to instruct the camera mapping module to turn on the target camera.

[0246] The camera mapping module determines that the first target front camera is the camera 2 of the second device based on the identifier of camera 2 in the second task information and the mapping relationship. For the specific description of the mapping relationship, reference can be made to the relevant descriptions in S1015 and S1016, which will not be elaborated here. After the camera mapping module determines the first target camera, it can issue a task instruction to turn on the first target camera.

[0247] S1414: The camera mapping module sends a second enable success notification to the interconnection service process through the first service.

[0248] For the specific description of S1414, reference can be made to the relevant description in S1406, which will not be elaborated.

[0249] S1415: The camera mapping module of the first device sends a camera 2 start instruction to the camera HAL of the second device through the first service of the first device, the interconnection service process, the interconnection service process of the second device, and the multimedia framework.

[0250] Correspondingly, the camera HAL of the second device receives the camera 2 start instruction from the camera mapping module of the first device through the first service of the first device, the interconnection service process, the interconnection service process of the second device, and the multimedia framework. The camera 2 start instruction is used to indicate to turn on camera 2.

[0251] S1416: The camera HAL of the second device turns on camera 2.

[0252] For the specific description of S1416, reference can be made to the relevant description in S1408, which will not be elaborated.

[0253] S1417: The camera HAL of the second device sends a first start success notification to the camera mapping module of the first device through the multimedia framework, the interconnection service process, the interconnection service process of the first device, and the first service.

[0254] After the camera HAL of the second device turns on camera 2, it can send a first start success notification to the first device.

[0255] S1418: The camera mapping module of the first device sends a switch success notification to the shared collaborative application through the first service and the interconnection service process.

[0256] After the camera mapping module of the first device receives the first close success notification and the first start success notification, it can send a switch success notification to the shared collaborative application through the first service and the interconnection service process. Correspondingly, the shared collaborative application of the first device sends a switch success notification to the camera mapping module through the first service and the interconnection service process. The switch success notification is used to notify that the switch is successful.

[0257] At this point, the captured image of the mirror projection of the second device in the first device is switched to the image data collected by Camera 2.

[0258] Between S1418 and S1419, the user can also click the control for switching the camera, and the electronic device can then execute the process of switching the camera. The process of switching the camera can refer to the relevant description in S1401 - S1418 and will not be elaborated here.

[0259] S1419: In response to the second operation, the first application on the first device sends a request to the shared collaboration application to close the Mth camera.

[0260] During Figure 10 the execution, when the user clicks the second operation, Camera N has been opened, and images are currently being captured through Camera N.

[0261] Correspondingly, the shared collaboration application on the first device receives the request from the first application to close the Mth camera. The request to close the Mth camera is used to request the closure of Camera N, and the request to close the Mth camera may include the identifier (ID) of Camera N.

[0262] S1420: The shared collaboration application on the first device sends an Mth enabling request to the interconnection service process.

[0263] S1421: The interconnection service process on the first device sends an Mth request to switch to the virtual camera to the first service.

[0264] S1422: The first service on the first device sends the 2M task information to the camera mapping module.

[0265] S1423: The camera mapping module on the first device sends the 2M task instruction to the second service.

[0266] S1424: The camera mapping module on the first device sends an Mth enabling success notification to the interconnection service process through the first service.

[0267] S1425: The second service on the first device sends a camera N closing instruction to the camera HAL.

[0268] S1426: The camera HAL on the first device closes Camera N.

[0269] S1427: The camera HAL on the first device sends an Mth closing success notification to the camera mapping module through the second service.

[0270] Among them, the processes of S1419 - S1427 can refer to the relevant description in S1401 - S1409 and will not be elaborated here.

[0271] S1428: The camera mapping module of the first device sends an end success notification to the shared collaboration application through the first service and the interconnected service process.

[0272] Correspondingly, the shared collaboration application of the first device receives the end success notification from the camera mapping module through the first service and the interconnected service process. The end success notification is used to notify the end of the process of collecting images by the camera.

[0273] In the above multi-screen collaboration process, whether the camera switches too fast or in a specified order, it will cause a large backlog of switching tasks, long response time, and at the same time pose a huge test to the stable operation of the device, which may cause the device to freeze or even black out, resulting in poor user experience. Therefore, in the cross-device camera collaboration process, frequent switching of the camera is likely to cause concurrent risks, resulting in a decrease in the switching success rate, etc.

[0274] In view of the above problems, the present application proposes a cross-device camera collaboration method and related devices. During the process of the user making frequent switches, the electronic device can screen and process multiple accumulated camera switching tasks, reduce multiple intermediate switching processes, determine the target camera, and directly switch to the target camera, reducing unnecessary switching processes, improving the switching efficiency, reducing the switching time, and also reducing the risks of freezing, picture freezing, and blacking out of the electronic device caused by frequent switching. Figure 15 It is another schematic diagram of the camera switching process shown in the embodiment of the present application. Exemplarily, as Figure 15 shown, different from Figure 13 , in response to 4 consecutive frequent switching operations at times T1, T2, T3, and T4, the electronic device can start from time T10 and directly switch the camera for collecting the captured picture from camera 1 to camera 5 without going through several intermediate switching processes as in Figure 13 . During the period from T10 to T9, the electronic device captures the content of the picture through camera 5. In addition, T0, T2, T3, T4, and T9 can all refer to the description of Figure 13 and will not be elaborated. In this way, the switching efficiency can be improved, the switching time can be reduced, and the risks of freezing and blacking out of the electronic device caused by frequent switching can also be reduced, improving the stability of the operation of the electronic device.

[0275] Optionally, before T0, when the electronic device accumulatively receives a switching operation, it can directly process this switching operation, that is, use camera 1 to capture the content of the picture in the shooting window.

[0276] Optionally, before T0, if the electronic device has received multiple switching operations, the multiple switching operations can be screened and processed. The last operation among the multiple switching operations is used to switch to Camera 1, and the electronic device can use Camera 1 to capture the picture content of the shooting window.

[0277] It should be noted that Figure 13 and Figure 15 The switching order in

[0278] Figure 16 is only an exemplary case. The number of switches (greater than 2) and the order are not limited. Figure 9 is a schematic diagram of a software architecture exemplarily provided by an embodiment of the present application. Combining the software architecture shown in

[0279] The interconnection service process in the application framework layer may further include: a DMSDP (interconnection service process) interface layer and a DMSDP system relay layer. The DMSDP interface layer is the data transmission interface of the interconnection service process, receives messages from the application layer, and sends the received data to the DMSDP system relay layer. The DMSDP system relay layer can manage the virtual camera service. For example, enable the virtual camera service.

[0280] The camera virtual adaptation process further includes: a task manager, a task processing module, and a session management module. The local device stores a task queue for camera switching, and each module in the camera virtual adaptation process can process the task queue. The task queue includes tasks for starting and closing the camera switch.

[0281] Among them, when the task manager receives task information, it creates a new camera task and sends the camera task to the task processing module. The task processing module is used to manage the tasks in the task queue. For example, adding a new camera task to the task queue.

[0282] The session management module can be used to process each task in the task list. The session management module can screen and process the tasks in the task queue to reduce the number of tasks to be processed. For example, the session management module copies tasks (the number of copied tasks is at least 1) from the task queue to the task list and processes the tasks in the task list. After the tasks in the task list are processed, the session management module can send a task indication to the camera mapping module to specifically execute the tasks in the task list.

[0283] In addition, Figure 16 For other modules in the software architecture module in Figure 9 reference can be made to the relevant description in

[0284] Figures 17A to 17CIt is a schematic flowchart of a method for camera switching in cross-device collaboration exemplarily provided by an embodiment of the present application. As Figures 17A to 17C shown, the method for cross-device multi-camera collaboration may include but is not limited to the following steps:

[0285] Figure 17A It is the processing procedure for starting the camera for the first time during the device collaboration process:

[0286] Among them, Figure 17A During the processing procedure, the camera collaboration configuration process executed in Figure 10 will also be performed (all steps in Figure 10 except for S1011~S1013), which will not be elaborated.

[0287] S1701: In response to the first operation, the first application of the first device sends a request to start the first camera to the shared collaboration application.

[0288] Correspondingly, the shared collaboration application receives the request to start the first camera from the first application. The request to start the first camera is used to request the start of Camera 1, and the request to start the first camera may include the identifier (ID) of Camera 1. The first operation may refer to the description of the first operation in S1001, which will not be elaborated.

[0289] S1702: The shared collaboration application of the first device sends a first enabling request to the DMSDP interface layer.

[0290] Correspondingly, the interconnected service process can receive the first enabling request from the shared collaboration application. The first enabling request may indicate enabling the virtual camera service, and the first enabling request may include the identifier of Camera 1, indicating to start Camera 1.

[0291] S1703: The DMSDP interface layer of the first device sends a first enabling request to the DMSDP system relay layer.

[0292] After receiving the first enabling request from the shared collaboration application, the DMSDP interface layer can send the first enabling request to the DMSDP system relay layer. Correspondingly, the DMSDP system relay layer can receive the first enabling request from the DMSDP interface layer.

[0293] After receiving the first enabling request, the DMSDP system relay layer can enable the virtual camera service.

[0294] S1704: The DMSDP system relay layer of the first device sends a first request to switch to the virtual camera to the first service.

[0295] Correspondingly, the first service can receive a first request to switch to a virtual camera from the relay layer of the DMSDP system in the interconnected service process. After the relay layer of the DMSDP system receives the first request to switch to a virtual camera, the relay layer of the DMSDP system can enable the virtual camera service. The first request to switch to a virtual camera may include the identifier of camera 1 to indicate the activation of camera 1.

[0296] S1705: The first service of the first device sends first task information to the task manager.

[0297] After the first service receives the first request to switch to a virtual camera, it can send the first task information to the task manager. Correspondingly, the task manager can receive the first task information from the first service. The first task information is used to indicate the activation of camera 1.

[0298] S1706: The task manager of the first device creates task 1.

[0299] After the task manager receives the first task information, it creates a task (task 1) to activate camera 1.

[0300] S1707: The task manager of the first device sends task information 1 to the task processing module.

[0301] After the task manager creates task 1, it can send task information 1 to the task processing module. Correspondingly, the task processing module receives task information 1 from the task manager. Task information 1 may include the information of task 1.

[0302] S1708: The task manager of the first device sends a first task addition notification to the relay layer of the DMSDP system through the first service.

[0303] Correspondingly, the relay layer of the DMSDP system receives the first task addition notification from the task manager through the first service. The first task addition notification is used to indicate that task 1 has been successfully added to the task list.

[0304] The task manager of the first device can also send a first enabling success notification to the relay layer of the DMSDP system through the first service. The process of sending the first enabling success notification can refer to the relevant content of S1406 and will not be elaborated here.

[0305] S1709: The task processing module of the first device adds task 1 to the task queue.

[0306] After the task processing module receives task information 1, it can add task 1 to the task queue. Task 1 is a task to activate camera 1.

[0307] The first device stores a task queue, and the task queue includes two types of camera processing tasks to be processed: a task of closing the camera and a task of opening the camera.

[0308] S1710: The task processing module of the first device sends a first task addition notification to the session management module.

[0309] After the task processing module adds task 1 to the task queue, it can send a first task addition notification to the session management module. Correspondingly, the session management module can receive the first task addition notification from the task processing module. The first task addition notification is used to indicate that a new task has been added to the task list.

[0310] S1711: The session management module of the first device copies the tasks in the task queue, obtains a first new task list, and clears the task queue.

[0311] When there is no task in the task list to process, the session management module can copy all the tasks in the task list in the first task, obtain the first new task list, and clear the task queue. At this point, there is no task in the task queue.

[0312] In the process of task copying, the session management module needs to lock the current task list first and then copy it. After the copy is completed, the session management module clears the task list and unlocks the task list. When the task list is locked, other modules cannot add, delete or change tasks.

[0313] S1712: The session management module of the first device filters and processes the first new task list.

[0314] In a possible implementation, the session management module may directly filter the tasks in the task list to obtain a simplified task list for processing.

[0315] The session management module of the first device can process the tasks that meet the preset conditions in the first new task list, and not process the tasks that do not meet the preset conditions, and determine that the tasks have been processed. The preset condition is that the task time in the task list is the earliest or latest condition.

[0316] In the embodiment of the present application, corresponding to: the task of closing the current camera and the task of opening the target camera, wherein the task of closing the current camera and the task of opening the target camera.

[0317] Figure 18 It is a flowchart of a method for task screening and processing disclosed exemplarily in an embodiment of the present application.

[0318] like Figure 18As shown, the method for screening and processing a task list may include but is not limited to the following steps: S1801: The session management module determines whether the number of tasks in the current task list is greater than 2. If the number of tasks in the first new task list is greater than 2, S1803 is executed. If the number of tasks in the first new task list is less than or equal to 2, S1802 is executed.

[0319] In S1701 - S1712, during the cross-device collaboration process, when the user first turns on the camera for the first time, the number of tasks is 1. Therefore, S1802 is executed.

[0320] S1802: The session management module directly processes the tasks in the first new task list.

[0321] In S1802, all tasks in the task list are directly processed.

[0322] S1803: The session management module determines whether the first task in the first new task list meets a preset condition. If the preset condition is met, the first task is retained in the second new task list; otherwise (when the first task does not meet the preset condition), the first task is not processed, and it is determined that the first task is completed.

[0323] Among them, the first task is the task traversed in sequence in the first new task list. For example, assume that the first new task list includes task 1, task 1,..., task 2n. The first task can be task 1, task 1,..., task 2n in sequence.

[0324] Exemplarily, task i and its time information are obtained in chronological order. When the time of task i is the earliest or the latest, task i is processed, and a task instruction is sent to the camera mapping module; when the time of task i is neither the earliest nor the latest, task i is discarded (not processed), and the session management module can send a task completion notification to the camera mapping module, indicating that task i is completed. Correspondingly, the camera mapping module can send a camera processing success notification to the shared collaboration application.

[0325] Exemplarily, the session management module sequentially determines the types of tasks in the first task list in chronological order, determines that the tasks with the earliest and latest times need to be executed, and the remaining tasks are not processed, and returns a processing completion notification. The tasks that need to be processed form the second new task list and are processed in sequence.

[0326] It should be noted that the number of tasks in the second new task list is less than or equal to the number of tasks in the first new task list. If a certain task in the first new task list is removed, the session management module can send a task completion notification to the camera mapping module. The task completion notification indicates that this task has been processed, such as the first startup success notification. Further, the camera mapping module can send a camera processing success notification to the shared collaborative application. For example, the camera 1 startup success notification.

[0327] S1713: The session management module of the first device sends a first task instruction to the camera mapping module.

[0328] Correspondingly, the camera mapping module can receive the first task instruction from the session management module.

[0329] S1714: The camera mapping module of the first device determines the first target camera based on the first task instruction and the mapping relationship.

[0330] After receiving the first task instruction, the camera mapping module can determine that the first pre-switching camera is the camera 1 of the first device based on the identifier of the camera 1 in the first task information and the mapping relationship. Among them. For the specific description of the mapping relationship, reference can be made to the relevant descriptions in S1015 and S1016, which will not be elaborated here. After the camera mapping module determines the first pre-switching camera, it can issue a task instruction to turn off the pre-switching camera.

[0331] S1715: The camera mapping module of the first device sends a camera 1 startup instruction to the camera HAL through the second service.

[0332] Correspondingly, the camera HAL can receive the camera 1 startup instruction from the camera mapping module through the second service.

[0333] S1716: The camera HAL of the first device starts the camera 1.

[0334] Among them, the description of S1716 can refer to the relevant description of S1012 and will not be elaborated.

[0335] So far, in response to the first operation, the user interface of the second device displayed on the first device starts to display a shooting window, and the shooting window displays the picture content collected by the camera 1.

[0336] S1717: The camera HAL of the first device sends a first startup success notification to the camera mapping module through the second service.

[0337] After the camera HAL starts camera 1, it can send a first startup success notification to the camera mapping module through the second service. Correspondingly, the camera mapping module can receive the first startup success notification from the camera HAL through the second service. Among them, the first startup success notification can indicate that camera 1 has started successfully.

[0338] S1718: The camera mapping module of the first device sends a camera 1 startup success notification to the shared collaborative application through the task manager, the first service, and the DMSDP interface layer.

[0339] Correspondingly, the shared collaborative application can receive the camera 1 startup success notification from the camera mapping module through the task manager, the first service, and the DMSDP interface layer.

[0340] So far, in the cross-device collaboration process, when the user clicks to start the camera for the first time and adds the task of starting the camera to the task list, since there is no task to be processed by the session management module currently, the task can be immediately copied and processed. During the process of the session management module processing the task of starting the camera, the user can continue to click to switch the camera. Multiple clicks will cause the tasks in the task list to pile up.

[0341] Regarding the accumulation of the number of tasks in the task list, it means Figure 17B In the process where the user switches the camera multiple times and processes the switching tasks:

[0342] S1719: In response to the first switching operation, the first application of the first device sends a request to close the first camera to the shared collaborative application.

[0343] When the display interface of the first device includes a control for the current camera switching function, the first switching operation is the operation of the user touching the camera switching control. Specifically, it can refer to the operations in Figure 11 and Figure 12 The description of the first switching operation can refer to the relevant description in S1401 and will not be elaborated here.

[0344] Among them, S1719 can refer to the relevant descriptions in S1401 and S1701 and will not be elaborated here.

[0345] S1720: The shared collaborative application of the first device sends a second enable request to the DMSDP interface layer.

[0346] S1721: The DMSDP interface layer of the first device sends a second enable request to the DMSDP system relay layer.

[0347] S1722: The DMSDP system relay layer of the first device sends a second request to switch to the virtual camera to the first service.

[0348] S1723: The first service of the first device sends second task information to the task manager.

[0349] S1724: The task manager of the first device creates Task 2.

[0350] S1725: The task manager of the first device sends task information 2 to the task processing module.

[0351] S1726: The task manager of the first device sends a second task addition notification to the DMSDP system relay layer through the first service.

[0352] S1727: The task processing module of the first device adds Task 2 to the task queue.

[0353] S1728: The task processing module of the first device sends a second task addition notification to the session management module.

[0354] Among them, the processing procedures of S1719 to S1728 can refer to the relevant content of S1701 to S1710 and will not be elaborated here.

[0355] S1729: In response to the first switching operation, the first application of the first device sends a request to start the second camera to the shared collaboration application.

[0356] S1730: The shared collaboration application of the first device sends a third enabling request to the DMSDP interface layer.

[0357] S1731: The DMSDP interface layer of the first device sends a third enabling request to the DMSDP system relay layer.

[0358] S1732: The DMSDP system relay layer of the first device sends a second switch-to-virtual-camera request to the first service.

[0359] S1733: The first service of the first device sends third task information to the task manager.

[0360] S1734: The task manager of the first device creates Task 3.

[0361] S1735: The task manager of the first device sends task information 3 to the task processing module.

[0362] S1736: The task manager of the first device sends a third task addition notification to the DMSDP system relay layer through the first service.

[0363] S1737: The task processing module of the first device adds Task 3 to the task queue.

[0364] S1738: The task processing module of the first device sends a third task addition notice to the session management module.

[0365] Among them, the processing procedures of S1729 to S1738 can refer to the relevant content of S1701 to 1710 and S1719 to S1728, which will not be elaborated here.

[0366] Between S1738 and S1739, the user can also click the control for switching the camera until S1719. In total, the electronic device can obtain H switching operations, where H is an integer greater than or equal to 0.

[0367] S1739: In response to the Kth switching operation, the first application of the first device sends a request to close the Kth camera to the shared collaboration application.

[0368] In S1739, starting from the first switching operation, the first device has received K switching operations in total.

[0369] S1740: The shared collaboration application of the first device sends a 2Kth enabling request to the DMSDP interface layer.

[0370] S1741: The DMSDP interface layer of the first device sends a 2Kth enabling request to the DMSDP system relay layer.

[0371] S1742: The DMSDP system relay layer of the first device sends a third request to switch to the virtual camera to the first service.

[0372] S1743: The first service of the first device sends the 2Kth task information to the task manager.

[0373] S1744: The task manager of the first device creates task 2K.

[0374] S1745: The task manager of the first device sends task information 2K to the task processing module.

[0375] S1746: The task manager of the first device sends a 2Kth task addition notice to the DMSDP system relay layer through the first service.

[0376] S1747: The task processing module of the first device adds task 2K to the task queue.

[0377] S1748: The task processing module of the first device sends a 2Kth task addition notice to the session management module.

[0378] S1749: In response to the first switching operation, the first application of the first device sends a request to start the (K + 1)th camera to the shared collaboration application.

[0379] S1750: The sharing and collaboration application of the first device sends a 2K + 1 enabling request to the DMSDP interface layer.

[0380] S1751: The DMSDP interface layer of the first device sends a 2K + 1 enabling request to the DMSDP system relay layer.

[0381] S1752: The DMSDP system relay layer of the first device sends a third request to switch to the virtual camera to the first service.

[0382] S1753: The first service of the first device sends 2K + 1 task information to the task manager.

[0383] S1754: The task manager of the first device creates task 2K + 1.

[0384] S1755: The task manager of the first device sends task information 2K + 1 to the task processing module.

[0385] S1756: The task manager of the first device sends a 2K + 1 task addition notification to the DMSDP system relay layer through the first service.

[0386] S1757: The task processing module of the first device adds task 2K + 1 to the task queue.

[0387] S1758: The task processing module of the first device sends a 2K + 1 task addition notification to the session management module.

[0388] Among them, the processing procedures of S1739 to S1758 can refer to the relevant content of S1719 to S1738 and will not be elaborated.

[0389] Between S1738 and S1739, the user can also click the control to switch the camera to S1719. The electronic device can obtain a total of K switching operations, where K is an integer greater than or equal to 2.

[0390] During the process of the session management module processing the previous tasks, the K switching operations of the user on the electronic device accumulate 2K processing tasks in the task list. When the session management module finishes processing the tasks, it reads the 2K tasks in the task list for copying and processing. The process of copying and processing is described in S1759 to S1773:

[0391] S1759: The session management module of the first device copies the tasks in the task queue to obtain a third new task list and clears the task queue.

[0392] Among them, the description of S1759 can specifically refer to the relevant description of S1711 and will not be elaborated.

[0393] S1760: The session management module of the first device performs screening processing on the third new task list.

[0394] Among them, for the description of S1760, specific reference can be made to the relevant description in S1712 Figure 18 which will not be elaborated here.

[0395] Combined with Figure 18 the process, if it is determined that the number of tasks 2K is greater than 2, the session management module can generate a fourth task list from the first camera closing task and the last camera opening task in the third new task list. The remaining tasks are discarded.

[0396] Optionally, after discarding a task, the session management module can send a processing success notification to the camera mapping module: for example, a closing success notification or a startup success notification.

[0397] S1761: The session management module of the first device sends a second task instruction to the camera mapping module.

[0398] The second task instruction is used to indicate to close Camera 1.

[0399] S1762: The camera mapping module of the first device determines a second target camera based on the second task instruction and the mapping relationship.

[0400] S1763: The camera mapping module of the first device sends a camera 1 closing instruction to the camera HAL through the second service.

[0401] S1764: The camera HAL of the first device closes Camera 1.

[0402] S1765: The camera HAL of the first device sends a first closing success notification to the camera mapping module through the second service.

[0403] S1766: The camera mapping module of the first device sends a camera 1 closing success notification to the shared collaborative application through the task manager, the first service, and the DMSDP interface layer.

[0404] Among them, the relevant content of S1760~S1766 can refer to the corresponding relevant descriptions of S1712~S1718, which will not be elaborated here.

[0405] S1767: The session management module of the first device sends a third task instruction to the camera mapping module.

[0406] The third task instruction is an instruction used to indicate to start Camera N.

[0407] S1768: The camera mapping module of the first device determines a third target camera based on the third task instruction and the mapping relationship.

[0408] Among them, the relevant description of S1768 can refer to the relevant descriptions of S1762 and S1714, which will not be elaborated here.

[0409] S1769: The camera mapping module of the first device sends a camera N start instruction to the camera HAL of the second device through the task manager, the DMSDP system relay layer, the interconnection service process of the second device, and the multimedia framework.

[0410] S1770: The camera HAL of the second device starts camera N.

[0411] S1771: The camera HAL of the second device sends a second start success notification to the camera mapping module through the interconnection service process of the second device, the DMSDP system relay layer of the first device, and the first service.

[0412] S1772: The camera mapping module of the first device sends a camera N start success notification to the shared collaborative application through the task manager, the first service, and the DMSDP interface layer.

[0413] Among them, the content of S1769~S1772 can refer to the relevant descriptions of S1415~S1418, which will not be elaborated here.

[0414] It should be noted that the above switching result is to switch from a physical camera to a virtual camera, or it can also be switched from a virtual camera to a physical camera, from one physical camera to another physical camera, or from one virtual camera to another virtual camera. The type of the target camera for switching and the current camera type are not limited.

[0415] In the above implementation, during the camera switching process of cross-device collaboration, the electronic device can simplify the processing process of multiple high-frequency switches, skip the intermediate (or part of the intermediate) switching process, reduce the number of switches, improve the switching efficiency, and reduce the switching time. Further, it will also avoid the excessive calculation caused by overly high-frequency continuous switching, thereby avoiding freezing or black screen, and increasing the stability and reliability of the camera switching during the collaboration process.

[0416] Such as Figure 17C As shown, in the case where the user wants to end the current collaborative shooting process, the user can click to turn off the camera. In response to the user's operation of turning off the camera, the following processes of S1773~S1790 are processed:

[0417] S1773: In response to the second operation, the first application of the first device sends a start first camera request to the shared collaborative application.

[0418] Among them, the relevant description of the second operation can refer to the relevant operations of S1419, which will not be elaborated here.

[0419] S1774: The sharing and collaboration application of the first device sends the H enabling request to the DMSDP interface layer.

[0420] S1775: The DMSDP interface layer of the first device sends the H enabling request to the DMSDP system relay layer.

[0421] S1776: The DMSDP system relay layer of the first device sends the fourth switch to virtual camera request to the first service.

[0422] S1777: The first service of the first device sends the H task information to the task manager.

[0423] S1778: The task manager of the first device creates task H.

[0424] S1779: The task manager of the first device sends the task information H to the task processing module.

[0425] S1780: The task manager of the first device sends the H task addition notification to the DMSDP system relay layer through the first service.

[0426] S1781: The task processing module of the first device adds task H to the task queue.

[0427] S1782: The task processing module of the first device sends the H task new addition notification to the session management module.

[0428] S1783: The session management module of the first device copies the tasks in the task queue to obtain the fifth task list and clears the task queue.

[0429] S1784: The session management module of the first device simplifies the fifth task list to obtain the sixth task list.

[0430] S1785: The session management module of the first device sends the fourth task indication to the camera mapping module.

[0431] S1786: The camera mapping module of the first device determines the fourth target camera based on the fourth task indication and the mapping relationship.

[0432] Among them, the relevant descriptions of S1773 to S1786 can refer to the relevant information of S1701 to S1714, which will not be elaborated here.

[0433] S1787: The camera mapping module of the first device sends the camera N shutdown instruction to the camera HAL through the task manager, the first service, the DMSDP system relay layer, the second device interconnection service process, and the multimedia framework.

[0434] The camera mapping module of the first device sends a camera N shutdown instruction to the camera HAL of the second device through the task manager, the DMSDP system relay layer of the second device, the interconnection service process of the second device, and the multimedia framework.

[0435] S1788: The camera HAL of the second device shuts down camera N.

[0436] S1789: The camera HAL of the second device sends a third shutdown success notification to the camera mapping module through the interconnection service process of the second device, the DMSDP system relay layer of the first device, and the first service.

[0437] S1790: The camera mapping module of the first device sends a camera N shutdown success notification to the shared collaboration application through the task manager, the first service, and the DMSDP interface layer.

[0438] The content of S1787 to S1790 above can refer to the relevant descriptions of S1769 to S1772 and will not be elaborated here.

[0439] From the above Figures 17A to 17C it can be seen that the number of switching operations that the first device can accumulate is uncertain. It can be 1 or multiple. If it is multiple, the intermediate camera tasks need to be skipped and the last two camera tasks need to be executed to ensure the efficiency and stability of task processing, reduce the occurrence of stuttering and black screens, and improve the user experience.

[0440] Figures 19A to 19D is a schematic diagram of a task processing process proposed in an embodiment of this application. Combining Figures 17A to 17C and Figure 18 as shown, the change process of the task list is described as follows:

[0441] When multiple devices are in the collaboration process, combined with the user's touch operation on the camera, the change process of the task queue and the task list is described as follows:

[0442] As Figure 19A shown, when the user clicks to turn on the camera, the task manager obtains the request to start camera 1 and can create a task to start camera 1. The task processing module adds the task to start camera 1 to the task queue. Since the session management module has no task to process, it can scan the task queue, copy the task to start camera 1 to the task list, and empty the tasks in the task queue.

[0443] As Figure 19B shown, during the process of the session management module processing the above task to start camera 1, when the user clicks to switch to camera 2, the task manager creates a task to turn off camera 1 and a task to start camera 2, and the task processing module adds these two tasks to the task queue.

[0444] As shown Figure 19C In the process of the session management module processing the task of starting the camera 1, the user clicks the camera switching operation multiple times. In the fourth switching operation of the user, the user clicks to switch to the camera 2, and the corresponding task manager creates the tasks of closing the camera 4 and starting the camera 5. The task processing module adds these two tasks to the task queue. The current task list accumulates tasks: closing the camera 1, starting the camera 2, closing the camera 2, starting the camera 3, closing the camera 3, starting the camera 4, closing the camera 4, and starting the camera 5. The above tasks are all copied to the task list.

[0445] As shown Figure 19D In the figure, the session management module simplifies the tasks in the task list, eliminates the tasks of starting the camera 2, closing the camera 2, starting the camera 3, closing the camera 3, starting the camera 4, and closing the camera 4. The remaining tasks of closing the camera 1 and starting the camera 5 are the tasks that need to be continued to be processed. This greatly reduces the accumulated number of tasks, thereby reducing unnecessary switching processes and improving the stability and efficiency of the cameras switched across devices.

[0446] Next, the device involved in the embodiments of the present application will be introduced.

[0447] Figure 20 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application.

[0448] The electronic device may include a processor 2010, an external memory interface 2020, an internal memory 2021, a Universal Serial Bus (USB) interface 2030, a charging management module 2040, a power management module 2041, a battery 2042, an antenna 1, an antenna 2, a mobile communication module 2050, a wireless communication module 2060, an audio module 2070, a microphone 2070C, a sensor module 2080, a camera 2093, and a display screen 2094, etc. Among them, the sensor module 2080 may include a touch sensor 2080K.

[0449] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0450] The processor 2010 may include one or more processing units. For example, the processor 2010 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and / or a Neural-network Processing Unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 2010 for storing instructions and data.

[0451] In the embodiments provided in this application, the electronic device may execute the shooting method through the processor 2010.

[0452] The internal memory 2021 may be used to store computer-executable program code, and the executable program code includes instructions. The processor 2010 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 2021.

[0453] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 2050 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device. The wireless communication module 2060 may provide solutions for wireless communications including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), etc. applied to the electronic device.

[0454] The electronic device realizes the display function through the GPU, the display screen 2094, the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 2094 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 2010 may include one or more GPUs, which execute program instructions to generate or change display information.

[0455] The display screen 2094 is used to display images, videos, etc. The display screen 2094 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a Micro LED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 2094, where N is a positive integer greater than 1. In the embodiments of the present application, if the electronic device 100 is the master device in the collaborative process, the screen displayed on the display screen may include the user interface of the collaborative device.

[0456] The electronic device can realize the acquisition function through the ISP, the camera 2093, the video codec, the GPU, the display screen 2094, the application processor, etc.

[0457] The ISP is used to process the data fed back by the camera 2093. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image or video visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 2093.

[0458] The camera 2093 is used to capture static images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a Charge Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image or video signal. The ISP outputs the digital image or video signal to the DSP for processing. The DSP converts the digital image or video signal into an image or video signal in standard RGB, YUV and other formats. In some embodiments, the electronic device may include multiple cameras 2093, and the number of cameras of the device is not limited in this application.

[0459] The video codec is used to compress or decompress digital videos. The electronic device can support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0460] The touch sensor 2080K, also known as the "touch panel". The touch sensor 2080K can be disposed on the display screen 2094, and the touch sensor 2080K and the display screen 2094 form a touch screen, also known as the "touch screen". The touch sensor 2080K is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 2094. In other embodiments, the touch sensor 2080K can also be disposed on the surface of the electronic device, at a different position from the display screen 2094.

[0461] In the embodiments of this application, the touch sensor 2080K can obtain the user's touch operation, for example, Figures 17A to 17C the first operation, the second operation, the switching operation, and so on.

[0462] It should be noted that, Figure 20 For the functions of other modules not mentioned in the electronic device shown, reference can be made to relevant technical documents, and this application will not elaborate on this.

[0463] As used in the foregoing embodiments, depending on the context, the term "when" may be interpreted to mean "if", "after", "in response to determining", or "in response to detecting". Similarly, depending on the context, the phrase "when determining" or "if detecting (the stated condition or event)" may be interpreted to mean "if determining", "in response to determining", "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0464] In the foregoing embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0465] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the foregoing method embodiments. The foregoing storage medium includes various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.

Claims

1. A camera collaboration method across devices, characterized in that, Applied to a first device, the first device is in a multi-screen collaboration state with a second device. The screen display interface of the first device is a first user interface, and the second user interface displayed in a first display area of the first user interface is the screen display picture of the second device. The method includes: At a first moment, the first device responds to a first operation, and a shooting window starts to be displayed on the second user interface, and the shooting window displays the picture content collected by a first camera. If n switching operations are cumulatively received at a second moment, the first device responds to the n switching operations and switches the camera for collecting the picture content from the first camera to a second camera. Wherein, the nth switching operation in the n switching operations is used to indicate switching the camera for collecting the picture content to a third camera; n is an integer greater than 1.

2. The method according to claim 1, characterized in that, After the second moment, the method further includes: If 1 switching operation is cumulatively received at a third moment, the first device responds to the 1 switching operation and switches the camera for collecting the picture content from the second camera to a third camera; the 1 switching operation is used to indicate switching the camera for collecting the picture content to the third camera, and the second camera and the third camera are different cameras.

3. The method according to claim 1 or 2, characterized in that, If m switching operations are cumulatively received from after the second moment to before a fourth moment, the method further includes: At the fourth moment, the first device responds to the m switching operations and switches the camera for collecting the picture content from the second camera to a fourth camera; the m switching operations are used to indicate switching the camera for collecting the picture content to the fourth camera, and the second camera and the fourth camera are different cameras, and m is an integer greater than 1.

4. The method according to claim 2, wherein If k switching operations are cumulatively received from after the third moment to before a fifth moment, the method further includes: At the fifth moment, the first device responds to the k switching operations and switches the camera for collecting the picture content from the third camera to a fifth camera; the k switching operations are used to indicate switching the camera for collecting the picture content to the fifth camera, and the second camera and the fifth camera are different cameras, and k is an integer greater than 1.

5. The method according to any one of claims 1-4, characterized in that, The first camera is a local camera of the first device; the second camera is a local camera of the first device or a camera of the second device.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: When the i-th switching operation is received, the first device creates an i-th camera closing task and an i-th camera starting task based on the i-th switching operation; and adds the i-th camera closing task and the i-th camera starting task to a task queue; the i-th switching operation is any one of the n switching operations, and i is an integer from 1 to n in sequence; The first device processes the first camera closing task and the nth camera starting task in the task queue.

7. The method according to claim 6, wherein The first device processes the first camera shutdown task and the nth camera startup task in the task queue, specifically including: When the task in the first task list is completed, the first device copies the tasks in the task queue to the second task list and clears the tasks in the task queue; Process the tasks in the second task list that meet the preset conditions, and do not process the tasks that do not meet the preset conditions; among them, the tasks that meet the preset conditions include the first camera shutdown task and the nth camera startup task.

8. The method according to claim 7, characterized in that The preset condition is that the task time in the second task list is the earliest or the latest.

9. The method according to claim 8, wherein The first device includes a task manager, a task processing module, and a session management module. The first device creates the first camera shutdown task and the first camera startup task based on the ith switching operation, specifically including: The task manager creates the first camera shutdown task and the first camera startup task based on the ith switching operation; Adding the first camera shutdown task and the first camera startup task to the task queue specifically includes: The task processing module adds the first camera shutdown task and the first camera startup task to the task queue; The first device processes the first camera shutdown task and the nth camera startup task in the task queue, specifically including: When all the tasks in the first task list are processed, the session management module copies all the tasks in the task queue to the second task list and performs a task screening and processing process on the second task list; after screening, the second task list only includes the first camera shutdown task and the nth camera startup task; The first device closes the first camera in response to the first camera shutdown task; and starts the second camera in response to the nth camera startup task.

10. The method according to claim 9, wherein The process of the task screening and processing is: Obtain the first task according to the traversal order of the second task list, determine whether the first task meets the preset condition, and if the first task meets the preset condition, process the first task; If the first task does not meet the preset condition, do not process the first task; Determine that the first task has been processed; When all the tasks in the second task list have been processed, end the task screening and processing process.

11. The method according to claim 9, wherein The process of the task screening and processing is: Add the tasks in the second task list that meet the preset conditions to the third task list, and process the tasks in the third task list in sequence; Discard the tasks in the second task list that do not meet the preset conditions.

12. An electronic device, characterized in that, Including: One or more processors and one or more memories; the one or more processors are coupled to the one or more memories, and the one or more memories are configured to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1-11.

Images