A cross-device camera collaboration method and apparatus

Abstract

Embodiments of the present application provide a kind of camera cooperation method and device across device, it is related to terminal field, can guarantee electronic equipment can call more kinds of camera across device, so that the camera called by electronic equipment is more in line with user demand.Its method is applied to first electronic equipment, first electronic equipment and second electronic equipment are in multi-screen cooperation state, and the method comprises: in response to the operation of user triggering video call, based on the first camera of first electronic equipment, first video data is collected;Second electronic equipment camera information is acquired;Wherein, camera information includes the number of the camera of second electronic equipment, the identification of camera and camera attribute, camera attribute includes orientation attribute, and orientation attribute includes front attribute or rear attribute;According to camera information, the second camera with the mapping relationship of first camera is determined;Based on the second camera of second electronic equipment, second video data is collected.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a cross-device camera collaboration method and apparatus. Background Technology

[0002] Multi-screen collaboration technology is a distributed technology that enables cross-device collaboration. Different devices can share resources and operate collaboratively. For example, after connecting two electronic devices (e.g., a mobile phone and a tablet), the tablet can display the mobile phone's desktop interface. The user can then operate the mobile phone on the desktop interface displayed on the tablet to perform corresponding functions. For instance, in response to a user triggering a video call function in an instant messaging application on the mobile phone's desktop interface displayed on the tablet, the mobile phone can initiate a video call.

[0003] In multi-screen collaboration scenarios, electronic devices can access cameras across devices, i.e., launch a virtual camera service (virtualized camera service). For example, when a mobile phone is making a video call, in addition to using its own camera to capture video frames, it can also switch to using the camera of a tablet to capture video frames.

[0004] Currently, electronic devices (e.g., mobile phones) can virtualize the use of default cameras (e.g., the front camera of a tablet) across devices, which does not fully meet the needs of users in real-world scenarios (e.g., a user wants to use the rear camera of a tablet, while the tablet defaults to using the front camera). Summary of the Invention

[0005] This application provides a cross-device camera collaboration method and apparatus, which can ensure that electronic devices can call up more types of cameras across devices, making the cameras called up by electronic devices more in line with user needs.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, a cross-device camera collaboration method is provided, applied to a first electronic device, wherein the first electronic device and a second electronic device are in a multi-screen collaboration state. The method includes: in response to a user triggering a video call, the first electronic device collects first video data based on its first camera, and displays a first video window based on the first video data; the first electronic device obtains camera information of the second electronic device; wherein the camera information includes the number of cameras of the second electronic device, the camera identifier, and camera attributes, the camera attributes including orientation attributes, and the orientation attributes including front-facing attributes or rear-facing attributes; the first electronic device determines a second camera that has a mapping relationship with the first camera based on the camera information; the second camera is a camera of the second electronic device; the first electronic device sends indication information to the second electronic device, the indication information being used to indicate the second camera; the first electronic device receives second video data from the second electronic device, the second video data being from the second camera; and the first electronic device displays a second video window based on the second video data.

[0008] Based on the method provided in this application embodiment, when a first electronic device and a second electronic device are in a multi-screen collaboration state, and a user triggers a video call on the first electronic device, the first electronic device can first collect first video data based on its first camera, then obtain the camera information of the second electronic device, and determine the second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device. The second camera is the camera of the second electronic device. Then, the second electronic device can be instructed to collect a video stream (second video data) based on the second camera, and the second video data can be obtained from the second electronic device to display a second video window. In other words, the first video data collected by the first camera is replaced with the second video data collected by the second camera. Thus, in a virtualized camera scenario (i.e., a scenario where the first electronic device (e.g., a mobile phone) calls the camera of the second electronic device (e.g., a tablet computer) across devices), the mobile phone can consider information such as the number and orientation attributes of the tablet computer's cameras when calling the camera across devices, enabling the mobile phone and tablet computer to perform camera collaboration more accurately.

[0009] In one possible implementation, the first electronic device determines a second camera that is mapped to the first camera based on camera information. This includes: when the second electronic device has two cameras, determining that the second camera mapped to the first camera is the one among the two cameras of the second electronic device that has the same orientation attribute as the first camera. That is, when the second electronic device (e.g., a tablet computer) has two cameras, the second camera can be determined based on its orientation attribute, and the orientation attribute of the second camera is the same as that of the first camera. This ensures that when the first electronic device (e.g., a mobile phone) calls a camera across devices, it can call a virtual camera with the same orientation attribute as the local camera (first camera), which better meets user needs. For example, when a mobile phone is making a video call, if the user uses the local front-facing camera to capture video frames on the phone, when switching to capturing video frames through the tablet computer's camera, the user can use the tablet computer's front-facing camera to capture video frames, making the camera switching more in line with the user's needs (the user's need to use the front-facing camera).

[0010] In one possible implementation, the first electronic device determines the second camera that is mapped to the first camera based on camera information. This includes: when the second electronic device has only one camera, determining that the first camera is mapped to that single camera of the second electronic device. That is, when the second electronic device has only one camera, the local camera is mapped to the tablet's single camera, without needing to consider the tablet's camera attributes. This ensures that the first electronic device (e.g., a mobile phone) can access the single camera of the second electronic device (e.g., a tablet) across devices, avoiding camera mapping failures that could lead to cross-device camera access failures.

[0011] In one possible implementation, the camera attributes also include tag attributes, which include at least one of wide-angle attributes, main camera attributes, telephoto attributes, or macro attributes.

[0012] In one possible implementation, the first electronic device determines a second camera that has a mapping relationship with the first camera based on camera information. This includes: when the number of cameras on the second electronic device is greater than two, determining that the second camera mapped to the first camera is the camera among the multiple cameras on the second electronic device that has the same tag attribute as the first camera. This ensures that when the first electronic device (e.g., a mobile phone) calls a camera across devices, it can call a virtual camera with the same tag attribute as the local camera (first camera), which better meets user needs. For example, when a mobile phone is making a video call, if the user uses the local wide-angle camera on the phone to capture video frames, when switching to capturing video frames through the camera on a tablet, the user can use the tablet's wide-angle camera, making the camera switching more in line with the user's need for a wide-angle camera.

[0013] In one possible implementation, the method further includes: if the second electronic device's multiple cameras do not include a camera with the same tag attribute as the first camera, the second camera that is mapped to the first camera is determined to be the camera among the multiple cameras of the second electronic device that has the same orientation attribute as the first camera. Thus, if the second camera mapped to the first camera cannot be determined based on the tag attribute, it can be determined based on the orientation attribute. This ensures that when the first electronic device (e.g., a mobile phone) calls a camera across devices, it can call a virtual camera with the same orientation attribute as the local camera (first camera), which better meets user needs.

[0014] In one possible implementation, the method further includes: the first electronic device acquiring the orientation attribute and / or label attribute of the first camera; the first electronic device determining a second camera with a mapping relationship to the first camera based on the camera information includes: the first electronic device determining the second camera with a mapping relationship to the first camera based on the camera information of the second electronic device and the orientation attribute and / or label attribute of the first camera. In this way, when a mobile phone calls a camera across devices, it can consider information such as the camera information of the second electronic device and the orientation attribute and / or label attribute of the first camera, enabling the mobile phone and tablet computer to perform camera collaboration more accurately.

[0015] Secondly, a cross-device camera collaboration method is provided, applied to a system including a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device are in a multi-screen collaboration state. The method includes: in response to a user triggering a video call, the first electronic device collects first video data based on its first camera, and displays a first video window based on the first video data; the first electronic device sends a request message to the second electronic device, the request message being used to request camera information of the second electronic device; the second electronic device receives the request message, collects camera information of the second electronic device, the camera information including the number of cameras of the second electronic device, camera identification, and camera attributes, the camera attributes including orientation attributes, the orientation attributes including front-facing attributes or back-facing attributes; the second electronic device sends its camera information to the first electronic device; the first electronic device receives its camera information from the second electronic device; the first electronic device determines a second camera that has a mapping relationship with the first camera based on the camera information; the second camera is the camera of the second electronic device; the first electronic device sends indication information to the second electronic device, the indication information being used to indicate the second camera; the second electronic device receives the indication information, and collects second video data based on the second camera based on the indication information; the second electronic device sends the second video data to the first electronic device; the first electronic device receives the second video data from the second electronic device; and the first electronic device displays a second video window based on the second video data.

[0016] Based on the method provided in this application embodiment, when a first electronic device and a second electronic device are in a multi-screen collaboration state, and a user triggers a video call on the first electronic device, the first electronic device can first collect first video data based on its first camera, then obtain the camera information of the second electronic device, and determine the second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device. The second camera is the camera of the second electronic device. Then, the second electronic device can be instructed to collect a video stream (second video data) based on the second camera, and the second video data can be obtained from the second electronic device to display a second video window. In other words, the first video data collected by the first camera is replaced with the second video data collected by the second camera. Thus, in a virtualized camera scenario (i.e., a scenario where the first electronic device (e.g., a mobile phone) calls the camera of the second electronic device (e.g., a tablet computer) across devices), the mobile phone can consider information such as the number and orientation attributes of the tablet computer's cameras when calling the camera across devices, enabling the mobile phone and tablet computer to perform camera collaboration more accurately.

[0017] In one possible implementation, the first electronic device determines a second camera that has a mapping relationship with the first camera based on camera information, including: when the number of cameras of the second electronic device is 2, determining that the second camera that has a mapping relationship with the first camera is the camera among the two cameras of the second electronic device that has the same orientation attribute as the first camera.

[0018] In one possible implementation, the first electronic device determines a second camera that has a mapping relationship with the first camera based on camera information, including: when the number of cameras of the second electronic device is 1, determining that the first camera has a mapping relationship with 1 camera of the second electronic device.

[0019] In one possible implementation, the camera attributes also include tag attributes, which include at least one of wide-angle attributes, main camera attributes, telephoto attributes, or macro attributes.

[0020] In one possible implementation, the first electronic device determines a second camera that has a mapping relationship with the first camera based on camera information, including: when the number of cameras of the second electronic device is greater than 2, determining that the second camera that has a mapping relationship with the first camera is the camera among the multiple cameras of the second electronic device that has the same tag attribute as the first camera.

[0021] In one possible implementation, the method further includes: if the multiple cameras of the second electronic device do not include a camera with the same label attribute as the first camera, determining that the second camera with a mapping relationship with the first camera is the camera with the same orientation attribute as the first camera among the multiple cameras of the second electronic device.

[0022] In one possible implementation, the method further includes: the first electronic device acquiring the orientation attribute and / or label attribute of the first camera; the first electronic device determining a second camera that has a mapping relationship with the first camera based on the camera information includes: the first electronic device determining a second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device and the orientation attribute and / or label attribute of the first camera.

[0023] Thirdly, a cross-device camera collaboration system is provided, comprising a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device are in a multi-screen collaboration state, wherein: the first electronic device is used to, in response to a user triggering a video call, acquire first video data based on a first camera of the first electronic device, and display a first video window based on the first video data; the first electronic device is also used to, send a request message to the second electronic device, the request message being used to request camera information of the second electronic device; the second electronic device is used to, receive the request message, acquire camera information of the second electronic device, the camera information including the number of cameras of the second electronic device, camera identifiers, and camera attributes, the camera attributes including orientation attributes, the orientation attributes including front-facing attributes or rear-facing attributes; the first... The second electronic device is further configured to: send camera information of the second electronic device to the first electronic device; receive camera information of the second electronic device from the second electronic device; determine, based on the camera information, a second camera that has a mapping relationship with the first camera, wherein the second camera is the camera of the second electronic device; send instruction information to the second electronic device, the instruction information being used to instruct the second camera; receive the instruction information and collect second video data based on the second camera according to the instruction information; send the second video data to the first electronic device; receive the second video data from the second electronic device; and display the second video data in a second video window based on the second video data.

[0024] Based on the method provided in this application embodiment, when a first electronic device and a second electronic device are in a multi-screen collaboration state, and a user triggers a video call on the first electronic device, the first electronic device can first collect first video data based on its first camera, then obtain the camera information of the second electronic device, and determine the second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device. The second camera is the camera of the second electronic device. Then, the second electronic device can be instructed to collect a video stream (second video data) based on the second camera, and the second video data can be obtained from the second electronic device to display a second video window. In other words, the first video data collected by the first camera is replaced with the second video data collected by the second camera. Thus, in a virtualized camera scenario (i.e., a scenario where the first electronic device (e.g., a mobile phone) calls the camera of the second electronic device (e.g., a tablet computer) across devices), the mobile phone can consider information such as the number and orientation attributes of the tablet computer's cameras when calling the camera across devices, enabling the mobile phone and tablet computer to perform camera collaboration more accurately.

[0025] Fourthly, this application provides a chip system including one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The aforementioned chip system can be applied to electronic devices including communication modules and memory. The interface circuits are used to receive signals from the memory of the electronic device and send the received signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device can perform the methods described in the first aspect and any of its possible design embodiments.

[0026] Fifthly, this application provides a computer-readable storage medium including computer instructions. When the computer instructions are executed on an electronic device (such as a mobile phone), they cause the electronic device to perform the methods described in the first aspect and any of its possible design embodiments.

[0027] Sixthly, embodiments of this application provide a cross-device camera collaboration device, including a processor and a memory coupled together. The memory stores program instructions, which, when executed by the processor, cause the device to implement the method described in the first aspect and any possible design of the above. The device may be an electronic device; or it may be a component of an electronic device, such as a chip.

[0028] In a seventh aspect, embodiments of this application provide a cross-device camera collaboration device. The device can be divided into different logical units or modules according to function, and each unit or module performs different functions so that the device performs the method described in the first aspect and any of its possible design methods.

[0029] Eighthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method as described in the first aspect and any possible design thereof.

[0030] It is understood that the beneficial effects achieved by the chip system described in the fourth aspect, the computer-readable storage medium described in the fifth aspect, the apparatus described in the sixth and seventh aspects, and the computer program product described in the eighth aspect can be referred to the beneficial effects in the first to third aspects and any possible design embodiments thereof, which will not be repeated here. Attached Figure Description

[0031] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;

[0032] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0033] Figure 3 This application provides a schematic diagram of a software architecture for calling an electronic device.

[0034] Figure 4 A schematic diagram of the software architecture of a invoked electronic device provided in an embodiment of this application;

[0035] Figure 5A This is a schematic diagram of the camera distribution of an electronic device provided in an embodiment of this application;

[0036] Figure 5B A schematic diagram of camera distribution in another electronic device provided in an embodiment of this application;

[0037] Figure 6 A schematic diagram of a tablet computer provided in an embodiment of this application;

[0038] Figure 7 A schematic diagram of a mobile phone display provided in an embodiment of this application;

[0039] Figure 8 A schematic diagram illustrating multi-screen collaboration provided in an embodiment of this application;

[0040] Figure 9 A schematic diagram illustrating yet another form of multi-screen collaboration provided in an embodiment of this application;

[0041] Figure 10 A schematic diagram illustrating yet another form of multi-screen collaboration provided in an embodiment of this application;

[0042] Figure 11 A schematic diagram of module interaction provided for an embodiment of this application;

[0043] Figure 12 This is a schematic diagram of another mobile phone provided in an embodiment of this application;

[0044] Figure 13 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0045] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first:

[0046] In multi-screen collaboration scenarios, electronic devices can access cameras across devices, i.e., launch a virtual camera service (virtualized camera service). For example, when a mobile phone is making a video call, in addition to using its own camera to capture video frames, it can also switch to using the camera of a tablet to capture video frames.

[0047] However, current virtual camera solutions have the following problems:

[0048] 1) In some solutions, electronic devices (e.g., mobile phones) can virtualize the use of a default camera (e.g., the front camera of a tablet) across devices, which does not meet the actual user needs in real-world scenarios (e.g., users want to use the rear camera of a tablet, while the tablet defaults to using the front camera).

[0049] 2) In other solutions, electronic devices can access cameras across devices based on their identifiers. For example, suppose a mobile phone is making a video call and uses its local camera with identifier 0 to capture video frames. When the phone switches cameras, it will access the camera with identifier 0 on the other end (e.g., a tablet). If the other end does not have a camera with identifier 0, the camera access will fail.

[0050] This application provides a cross-device camera collaboration method that can avoid cross-device camera call failures and ensure that electronic devices can call more types of cameras across devices, making the cameras called by electronic devices more in line with user needs.

[0051] Figure 1 An exemplary schematic diagram illustrates an application scenario to which embodiments of this application are applicable. For example... Figure 1 As shown, this application scenario includes electronic device 001 and electronic device 002. Electronic device 001 and electronic device 002 can be different types of electronic devices. For example, electronic device 001 can be a mobile phone, and electronic device 002 can be a tablet computer.

[0052] Figure 1 In this system, electronic devices 001 and 002 can be interconnected via a wireless communication network. This wireless communication network can be a local area network (LAN). For example, when the communication network is a LAN, it can be a short-range communication network such as a Wi-Fi hotspot network, a Wi-Fi P2P network, a Bluetooth network, a Zigbee network, or an NFC network. Figure 1 In the scenario shown, different electronic devices can send data through a communication network, such as sending pictures, text, videos, or search results of objects such as pictures, text, or videos.

[0053] It should be noted that the above system may also include more electronic devices, such as laptops, smart TVs, smartwatches, etc., and the embodiments of this application do not limit this.

[0054] The following description uses an electronic device (e.g., electronic device 001 or electronic device 002) as an example to illustrate the hardware structure of electronic device 100. Figure 2A schematic diagram of the hardware structure of electronic device 100 is shown. Electronic device 100 can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose special limitations on the specific type of electronic device.

[0055] See Figure 2 Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management process 140, power management process 141, battery 142, antenna 1, antenna 2, mobile communication process 150, wireless communication process 160, audio process 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor process 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor process 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0056] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0058] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0059] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0060] In some embodiments, the processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0061] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication process 150, wireless communication process 160, modem processor, and baseband processor.

[0062] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0063] The mobile communication process 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication process 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication process 150 can receive electromagnetic waves through antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication process 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0064] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194.

[0065] Wireless communication process 160 can provide solutions for wireless communication applications on electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. Wireless communication process 160 can be one or more devices integrating at least one communication processing process. Wireless communication process 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. Wireless communication process 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0066] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication process 150, and antenna 2 is coupled to wireless communication process 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology.

[0067] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0068] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's image sensor. The light signal is converted into an electrical signal, and the image sensor transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 193.

[0069] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0070] A digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0071] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0072] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external memory card.

[0073] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback), etc. The data storage area may store data created by electronic device 100 during use (such as audio data), etc.

[0074] The methods described in the following embodiments can all be implemented in the electronic device 100 having the above-described hardware structure.

[0075] The software system of the aforementioned electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of the electronic device 100.

[0076] The software architecture of electronic devices 001 and 002 is described below. The software architectures of electronic devices 001 and 002 differ. In multi-screen collaboration scenarios, electronic device 001 can act as the calling device (also known as the master device), and electronic device 002 can act as the called device (also known as the controlled device). Taking electronic device 001 as a mobile phone and electronic device 002 as a tablet computer as an example, after the mobile phone and tablet computer establish a collaborative connection, the tablet computer can display the mobile phone's desktop interface. The user can operate on the mobile phone's desktop interface displayed on the tablet computer to enable the mobile phone to perform corresponding functions.

[0077] Additionally, in scenarios involving shared cameras, after a mobile phone and tablet establish a collaborative connection, the tablet can access the phone's camera. In this case, the tablet is the accessing device, and the mobile phone is the accessed device.

[0078] Figure 3 This is a software architecture for a called / controlled device (e.g., electronic device 001) provided in an embodiment of this application. See also Figure 3A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into the application layer, application framework layer, system layer, extension layer, and kernel layer from top to bottom.

[0079] The application layer can include a series of application packages. For example... Figure 3 As shown, the application package can include, but is not limited to, applications such as instant messaging, multi-screen collaboration, camera sharing, camera, Bluetooth, gallery, calling, and maps. Among them, the instant messaging application can be used to enable video calls; the multi-screen collaboration application can be used to enable multi-screen collaboration functionality; and the camera sharing application can be used to enable cross-device camera access.

[0080] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 3 As shown, the application framework layer may include a service discovery process, a collaboration assistant process, and an interconnection service process.

[0081] The service discovery process listens for connection commands indicating collaboration after Bluetooth or NFC is enabled, and notifies the collaboration assistant process upon receiving the command. The collaboration assistant process, upon receiving the notification from the service discovery process, establishes a collaborative connection by exchanging information with collaboration assistant processes in other electronic devices.

[0082] As an example of this application, during a video call, in a collaborative scenario, the interconnect service process enables the virtualized camera service to establish a data transmission channel between the interconnect service process and the virtual camera process. Additionally, after successfully enabling the virtualized camera service, the interconnect service process also sends a video stream to the calling device (the master device).

[0083] In one example, the interconnect service process includes a data processing module, a transmission channel module, a flow control module, and a capability acquisition module. The data processing module processes video frames according to underlying requirements, such as format conversion; the transmission channel module configures the transmission channel; the flow control module buffers the video stream; and the capability acquisition module acquires the camera capabilities (i.e., camera information) of the local electronic device and sends these capabilities to the calling device (the main control device).

[0084] As an example of this application, the system layer includes an audio framework, a multimedia framework, etc.

[0085] The audio framework supports various common audio encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The multimedia framework supports various common video formats and still image files. Supported video formats include, for example, AVI, WMV, FLV, and Blu-ray.

[0086] As an example of this application, the extension layer may include a camera process and a virtual camera process, both residing in the hardware abstraction layer (HAL). The camera process is used to open the local camera according to the business requirements of the application layer and capture video frames through the local camera. In one embodiment, in a collaborative scenario, the camera process is used to send the video frames captured by the local camera to the interconnect service process according to the business requirements of the virtual camera process, so that the interconnect service process can then send the video frames captured by the local camera to the calling device (master device).

[0087] The kernel layer is the layer between hardware and software. It contains at least display drivers, camera drivers, audio drivers, and sensor drivers. The camera driver, in particular, powers the camera hardware, enabling the camera to start and allowing the electronic device to capture images.

[0088] Figure 4 This is a software architecture for calling a device / master device (e.g., electronic device 002) provided in an embodiment of this application. See also Figure 4 A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into the application layer, application framework layer, system layer, extension layer, and kernel layer from top to bottom.

[0089] The application layer can include a series of application packages. For example... Figure 4 As shown, the application package can include, but is not limited to, applications such as instant messaging, multi-screen collaboration, camera sharing, camera, Bluetooth, gallery, calling, and maps. Among them, the instant messaging application can be used to enable video calls; the multi-screen collaboration application can be used to enable multi-screen collaboration functionality; and the camera sharing application can be used to enable cross-device camera access.

[0090] The application framework layer provides application programming interfaces and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. For example... Figure 4 As shown, the application framework layer may include a service discovery process, a collaboration assistant process, and an interconnection service process.

[0091] The service discovery process listens for connection commands indicating collaboration after Bluetooth or NFC is enabled, and notifies the collaboration assistant process upon receiving the command. The collaboration assistant process, upon receiving the notification from the service discovery process, establishes a collaborative connection by exchanging information with collaboration assistant processes in other electronic devices.

[0092] As an example of this application, during a video call, in a collaborative scenario, the interconnect service process enables the virtualized camera service to establish a data transmission channel between the interconnect service process, the virtual camera adaptation process, and the virtual camera process. Furthermore, after successfully enabling the virtualized camera service, the interconnect service process also receives and caches the video stream sent by the called device (controlled device) and provides corresponding camera services to the underlying layer based on its requests.

[0093] In one example, the interconnect service process includes a data processing module, a transmission channel module, a flow control module, and a capability aggregation module. The data processing module is used to process video frames according to the underlying requirements, such as performing format conversion; the transmission channel module is used to configure the transmission channel; the flow control module is used to buffer the video stream; and the capability aggregation module is used to store the camera capabilities of the controlled device.

[0094] As an example of this application, the system layer includes a virtual camera adaptation process, a multimedia framework, etc.

[0095] The multimedia framework supports video formats such as AVI, WMV, FLV, and Blu-ray.

[0096] The virtual camera adaptation process can be used to perform format conversion processing on the data transmitted between the interconnect service process and the extension layer.

[0097] In one example, the virtual camera adaptation process includes a first service, a second service, a converter, a camera relationship mapping module, and a camera switching management module.

[0098] The first service is used to provide services to the upper layer and supports a first data format, such as the CHANNEL service. If the interconnect service process needs to send data to the underlying virtual camera process, it can first convert the data according to the first data format, and then send the converted data to the virtual camera adaptation process through the first service, so that the virtual camera adaptation process can send it to the virtual camera process.

[0099] The second service is used to provide services to lower-level processes and supports a second data format, such as the TRANSLATOR service. If the virtual camera process needs to send data to the upper-layer interconnect service process, it can first convert the data according to the second data format, and then send the converted data to the virtual camera adaptation process through the second service, so that the virtual camera adaptation process can send it to the interconnect service process.

[0100] A converter is used to convert data formats. For example, it can convert lower-level data into a data format that the upper-level layer can recognize and process, or vice versa.

[0101] The converter may include a camera mapping management module. This module maps the camera (or webcam) of the local electronic device to the camera of the controlled device. For example, if the collected camera capability information (webcam information) determines that both the local electronic device and the controlled device include a front-facing camera and a rear-facing camera, the converter maps the front-facing camera of the local electronic device to the front-facing camera of the controlled device, and the rear-facing camera of the local electronic device to the rear-facing camera of the controlled device.

[0102] As an example of this application, the extension layer mainly includes a camera process and a virtual camera process, both located in the HAL layer. The camera process is used to open the local camera according to the business requirements of the application layer and capture video frames through the local camera. In one embodiment, in a collaborative scenario, the camera process requests the video stream captured by the controlled device through the virtual camera process. Correspondingly, the virtual camera process, based on the request from the camera process, obtains the video frames captured by the controlled device from the interconnect service process and sends the video frames captured by the controlled device to the camera process to replace the local video stream, thereby achieving camera switching.

[0103] The kernel layer is the layer between hardware and software. It contains at least display drivers, camera drivers, audio drivers, and sensor drivers. The camera driver, in particular, powers the camera hardware, enabling the camera to start and allowing the electronic device to capture images.

[0104] The following is combined with Figures 5A-5B This document describes the camera distribution of various types of electronic devices (such as mobile phones, tablets, laptops, and foldable phones).

[0105] like Figure 5AAs shown in (a), a tablet computer may include two cameras: a front-facing camera and a rear-facing camera. For example, the IDs of the front-facing camera and the rear-facing camera on the tablet computer are 0 (which could also be Camera0) and 1 (which could also be Camera 1), respectively. Figure 5A As shown in (b) above, a mobile phone may include four cameras: a front-facing camera, a rear telephoto camera, a rear wide-angle camera, and a rear main camera. For example, the identifiers (IDs) for the front-facing camera, the rear telephoto camera, the rear wide-angle camera, and the rear main camera are 1, 0, 6, and 7, respectively. Figure 5A As shown in (c), a laptop computer may include a camera, such as a front-facing camera. For example, the front-facing camera on a laptop computer has an ID of 0. Figure 5B As shown in (a), a foldable phone may include a front-facing camera on the outer screen (third screen), a rear telephoto camera, a rear wide-angle camera, and a rear main camera. Figure 5B As shown in (b), the foldable phone may also include an inner screen front-facing camera. For example, the IDs of the inner screen front-facing camera, the rear main camera, the rear telephoto camera, the rear wide-angle camera, and the outer screen front-facing camera of the foldable phone are 1, 0, 9, 8, and 13, respectively.

[0106] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of this application, unless otherwise stated, "at least one" refers to one or more, and "more than one" refers to two or more. Furthermore, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0107] To make it easier to understand, the following example of collaboration between mobile phones and tablets will be used to introduce several possible connection methods for multi-screen collaboration.

[0108] 1. Establish a connection via Bluetooth.

[0109] In one embodiment, when a user wants to work collaboratively with a mobile phone and tablet, they can enable Bluetooth and Wi-Fi on both the phone and tablet. Additionally, on the mobile phone, the user can manually enable the multi-screen collaboration function. For example, the user can find the "Multi-device Collaboration" switch through "Settings" - "More Connections" - "Multi-device Collaboration" and turn it on. Optionally, the mobile phone and tablet can be logged into the same account (e.g., a Honor account).

[0110] like Figure 6 As shown in (a), the user swipes down from the tablet's status bar to reveal a notification panel, which includes a "Multi-screen Collaboration" option 31. The user can tap the "Multi-screen Collaboration" option 31. In response to the user's triggering of the "Multi-screen Collaboration" option 31, the tablet displays a first prompt window, which includes first operation prompts instructing the user on how to operate. For example, as... Figure 6 As shown in (b) above, the first operation prompt includes the following: "1. Turn on your phone's Bluetooth and bring it close to the device. Once the device is detected, click 'Connect'. 2. After connecting, you can operate your phone on the tablet to achieve data sharing between devices." Thus, users can operate according to the first operation prompt in the first prompt window. For example, bringing the phone close to the tablet.

[0111] In one example, as the phone approaches the tablet, when the phone detects the tablet, it displays a second prompt window, such as... Figure 7 As shown in (a), the second prompt window includes a "Connect" option 41 and a "Cancel" option 42. In response to the user's triggering of the "Connect" option 41, the phone establishes a connection with the tablet via Bluetooth. In another example, when the phone is near the tablet, it may automatically establish a connection with the tablet via Bluetooth without displaying the second prompt window.

[0112] As an example, and not a limitation, during the process of a mobile phone establishing a connection with a tablet via Bluetooth, the phone can also display a third prompt window to indicate that the connection is in progress. For example, Figure 7 As shown in (b), the third prompt window can be window 43. Optionally, window 43 includes a "Cancel" option so that the user can cancel the connection during the connection establishment process if needed.

[0113] 2. Establish a connection by scanning a QR code.

[0114] On the tablet, users can follow the path "My Phone" - "Connect Now" - "Scan to Connect" to operate the connection. In response to the user's action, the tablet displays a QR code for establishing the connection. Optionally, the tablet can also display a second operation prompt to guide the user on how to operate, such as "Scan the QR code to connect using your mobile browser".

[0115] On a mobile device, users can access an interface with a "Scan" option through a browser (or Smart Vision). In response to the user's activation of the "Scan" option, the phone's camera is activated, allowing the user to point the camera at the QR code displayed on the tablet and scan it.

[0116] In one example, after the mobile phone successfully scans the QR code, it sends a connection request to the tablet. Upon receiving the request, the tablet can display a prompt message, such as "Device xx requests to establish a connection with this device. Do you agree to establish the connection?". Additionally, a fourth prompt window can include "Agree" and "Deny" options. In response to the user's "Agree" option, the tablet and mobile phone establish a connection.

[0117] It should be noted that the above explanation only illustrates opening the QR code on a tablet via the path "My Phone" - "Connect Now" - "Scan to Connect". In another embodiment, the QR code can also be opened via other paths. For an example, please refer to... Figure 6 In Figure (b), the first prompt window includes a first operation prompt and a second operation prompt: "Can't find this device? You can also connect by scanning the QR code." The phrase "connect by scanning the QR code" is set to triggerable. Users can click on the "connect by scanning the QR code" content in the first prompt window. In response to the user's triggering of the "connect by scanning the QR code" action, the tablet's interface displays a QR code. Users can then scan the QR code on the tablet with their mobile phones to establish a connection.

[0118] 3. Establish a connection by tapping.

[0119] Users can enable NFC and multi-screen collaboration on their phone and tablet respectively. Then, the user touches the NFC area on the back of the phone (around the rear camera) to the NFC area on the keyboard. In response to this action, the phone and tablet establish a connection via NFC. Optionally, before establishing the connection via NFC, the user can be prompted on both the tablet and the phone to agree to the connection. After the user agrees, the phone and tablet will perform the connection establishment process. In one example, when the phone and tablet are connected, the phone can also notify the user via vibration or ringing.

[0120] It should be noted that the above-described possible connection methods are all illustrated using wireless methods as examples. In another embodiment, screen projection can also be performed via a wired method, such as through a Type-C to High Definition Multimedia Interface (HDMI) cable. This application does not limit the scope of this embodiment.

[0121] After the phone and tablet are successfully connected, as follows: Figure 8 As shown, a window from a mobile phone can be mirrored on a tablet. This allows the user to perform operations within that window as needed.

[0122] In one example, when a user wants to make a video call via instant messaging software, they can click the icon of instant messaging software A in this window on the tablet to open the software, and then click the "Video Call" option within it. In response to the user's triggering of the "Video Call" option, the tablet sends a video call control command to the phone. Upon receiving the command, the phone initiates a video call request to begin a video call with the other user. Please refer to [reference needed]. Figure 9 During this process, the screens of the mobile phone and tablet are displayed synchronously.

[0123] In one embodiment, please refer to Figure 10 In response to a user pulling down the tablet's notification bar, the tablet can display a notification interface 1201. Notification interface 1201 may include a multi-screen collaboration notification 1202, which displays that the device is connected (collaborating) with the phone. Multi-screen collaboration notification 1202 may also include a disconnect button and an audio / video switch to phone button. Users can adjust the on / off state of these buttons as needed. It should be understood that when the tablet is connected (collaborating) with the phone, it can use the tablet's camera to capture video by default. If the user only wants to use the phone's own camera to capture video, they can operate the audio / video switch to phone button (e.g., click the audio / video switch to phone button), so that the video feed during a video call can be captured by the phone's camera.

[0124] The following example, using a mobile phone as the first electronic device and a tablet computer as the second, illustrates a method for cross-device camera collaboration between a mobile phone and a tablet computer, with the two devices establishing a collaborative connection (i.e., in a multi-screen collaboration state). The mobile phone can be the calling end (master end), and the tablet computer can be the called end (controlled end). Figure 11 As shown, the method includes:

[0125] 1101. The mobile phone's instant messaging application sends a request to the mobile phone's multi-screen collaboration application to open the camera.

[0126] The request to open the camera can include the identifier (ID) of the phone's local camera.

[0127] Multi-screen collaboration is established between mobile phones and tablets. When a user triggers a video call on the mobile phone, the mobile phone's instant messaging application sends a request to the mobile phone's multi-screen collaboration application to open the camera.

[0128] In some embodiments, after multi-screen collaboration is established between a mobile phone and a tablet computer, in response to a user triggering a video call on the mobile phone, the mobile phone's instant messaging application can send a request to the mobile phone's multi-screen collaboration application to open the camera. The action of triggering a video call can include initiating or answering a video call, and this application does not limit the scope of the request.

[0129] In other embodiments, when a video call has been initiated by the mobile phone, in response to the user triggering the establishment of multi-screen collaboration between the mobile phone and the tablet, the mobile phone's instant messaging application can send a request to the mobile phone's multi-screen collaboration application to open the camera.

[0130] Multi-screen collaboration applications can also be camera-sharing applications. Camera-sharing applications enable the use of cameras across different devices.

[0131] It should be noted that when establishing a multi-screen collaboration connection between a mobile phone and a tablet, and when the user triggers a video call on the mobile phone, steps 1101-1110 can be executed to obtain the tablet's camera information. Simultaneously, steps 1111-1114 can be executed to obtain the camera attributes of the currently active camera on the mobile phone. Finally, based on the camera attributes of the currently active camera on the mobile phone and the camera information on the tablet, a mapping and binding can be performed between the local camera (the camera of the calling electronic device (e.g., the mobile phone)) and the virtual camera (i.e., the camera of the called electronic device (e.g., the tablet)), which allows execution of step 1115. Afterwards, steps 1116-1121 can be executed.

[0132] 1102. The mobile phone's multi-screen collaboration application sends a request to the mobile phone's interconnection service process to enable the virtualized camera service.

[0133] The interconnect service process enables virtualized camera services to establish a data transmission channel between the interconnect service process and the virtual camera process.

[0134] 1103. The mobile phone's interconnection service process requests camera information from the tablet computer from the mobile phone's capability aggregation module.

[0135] 1104. The mobile phone's capability aggregation module requests camera information from the tablet computer's capability acquisition module.

[0136] The mobile phone's capability aggregation module can send a request message to the tablet's capability acquisition module through its own transmission channel module. This request message is used to request camera information from the tablet.

[0137] 1105. The tablet's capability acquisition module requests camera information from the tablet's multimedia framework.

[0138] The tablet's capability acquisition module can receive request messages from the mobile phone through its own transmission channel module. These request messages are used to request camera information from the tablet.

[0139] 1106. The multimedia framework of the tablet computer obtains information from the tablet computer's camera.

[0140] The tablet's camera information includes the identifiers and attributes of all available cameras on the tablet.

[0141] The camera identifier, also known as the camera number, is used to identify different cameras. Different cameras have different identifiers. For example, the IDs for the front-facing camera, rear telephoto camera, rear wide-angle camera, and rear main camera on a mobile phone are 1, 0, 6, and 7, respectively.

[0142] Camera attributes can include orientation attributes, which include front and back attributes.

[0143] For example, a tablet computer can call a first interface at the system layer to read the orientation attribute of each camera. This first interface can be, for example, `CameraCharacteristics.LENS_FACING`. For each camera on the tablet, the first interface can be called to read its orientation attribute based on its corresponding identifier (ID). If the first interface returns `CameraMetadata.LENS_FACING_FRONT`, it indicates that the camera is a front-facing camera, and its orientation attribute can be marked as "FRONT". If the first interface returns `CameraMetadata.LENS_FACING_BACK`, it indicates that the camera is a rear-facing camera, and its orientation attribute can be marked as "BACK".

[0144] In one possible design, camera attributes may also include tag attributes, which may include at least one of wide-angle attributes, main camera attributes, telephoto attributes, or macro attributes.

[0145] For example, a tablet computer can call a second interface at the system layer to read the tag attributes. This second interface could be, for example, `KeyGenerator.generateCharacteristicsKey`. For each camera on the tablet, the second interface can be called to read the camera's tag attributes based on its corresponding identifier (ID). If the return value (key value) of the second interface is `com.hihonor.device.capabilities.mainModeSupported`, the camera is considered a rear main camera, and its tag attribute is marked as the main camera attribute (Main). If the return value (key value) of the second interface is `com.hihonor.device.capabilities.teleSupported`, the camera is considered a telephoto camera, and its tag attribute is marked as the telephoto attribute (Focus). If the return value (key value) of the second interface is `com.hihonor.device.capabilities.wideModeSupported`, the camera is considered a wide-angle camera, and its tag attribute is marked as the wide-angle attribute (Wide).

[0146] Optionally, camera attributes may also include other attributes, such as magnification attributes and exposure attributes. For example, magnification attributes may include high magnification and low magnification attributes. Exposure attributes may include high exposure rate and low exposure rate attributes.

[0147] Optionally, the tablet's camera information may also include streaming capability information. This streaming capability information may include the bitrate supported by the camera (e.g., H.265 / H.264, etc.) and the resolution supported by the camera (e.g., 480P / 720P, etc.). All available cameras on the tablet may include both the tablet's built-in camera and external cameras; this application does not limit the scope.

[0148] For example, suppose the tablet computer can support two cameras, including Camera 1 (ID: Camera 1) and Camera 2 (ID: Camera 0). The camera information of the tablet computer can be shown in Table 1:

[0149] Table 1

[0150]

[0151] In this application, the orientation attribute of camera 1 is FRONT (i.e., front-facing), and the tag attribute is empty, meaning no tag attribute for camera 1 was found. Alternatively, the tag attribute of camera 1 can be wide-angle, telephoto, main camera, etc., which is not limited in this application. The orientation attribute of camera 2 is BACK (i.e., rear-facing), and the tag attribute is Main (i.e., main camera).

[0152] Camera information can be in JSON format. JSON is a lightweight data exchange format. For example, the camera information of a tablet computer is as follows:

[0153] {"id":"Camera1","streamAbility":{"Supported bitrate":{H265 / H264…},"Supported resolution":{480P / 720P…},"Other information":{"Orientation attribute":"Front"}};

[0154] {"id":"Camera0","streamAbility":{"Supported bitrate":{H265 / H264…},"Supported resolution":{480P / 720P…},"Other information":{"Tag attribute":"BACK"},{"Tag attribute":"Main"}}.

[0155] In one possible design, the tablet's camera information may also include the number of cameras on the tablet. For example, the tablet may have two cameras.

[0156] It should be understood that the embodiments of this application are illustrated using a tablet computer as the called end (controlled end). In actual applications, the called end (controlled end) can also be other types of electronic devices, such as mobile phones, PCs, etc. When the called end (controlled end) is a mobile phone or PC, the way the mobile phone or PC obtains its own camera information can refer to the way the tablet computer obtains camera information, and this application will not elaborate on it.

[0157] 1107. The multimedia framework of the tablet computer returns the camera information of the tablet computer to the tablet computer's capability acquisition module.

[0158] 1108. The tablet's capability acquisition module returns the tablet's camera information to the phone's capability aggregation module.

[0159] The tablet's capability acquisition module can send camera information from the tablet to the mobile phone via its own transmission channel module. The mobile phone can receive the camera information from the tablet via its own transmission channel module and can then send the tablet's camera information to the phone's capability aggregation module.

[0160] 1109. The mobile phone's capability summary module stores the tablet's camera information.

[0161] In one possible design, if the camera information from the tablet does not include the number of cameras on the tablet, the phone's capability aggregation module can determine the number of cameras on the tablet. For example, the phone's capability aggregation module can count the number of cameras on the tablet based on the number of camera identifiers carried in the tablet's camera information. For example, the camera identifiers carried in the tablet's camera information may include "Camera1" and "Camera0", meaning the number of cameras on the tablet can be 2.

[0162] 1110. The mobile phone's capability aggregation module returns the tablet's camera information to the mobile phone's camera mapping management module.

[0163] In some embodiments, after multi-screen collaboration is established between a mobile phone and a tablet computer, in response to a user triggering a video call on the mobile phone, the mobile phone's instant messaging application can send a request to the mobile phone's multi-screen collaboration application to open the camera. The action of triggering a video call can include initiating or answering a video call, and this application does not limit the scope of the request.

[0164] In other embodiments, when a video call has been initiated by the mobile phone, in response to the user triggering the establishment of multi-screen collaboration between the mobile phone and the tablet, the mobile phone's instant messaging application can send a request to the mobile phone's multi-screen collaboration application to open the camera.

[0165] 1111. The phone's instant messaging application sends a request to the phone's camera HAL to open the local camera.

[0166] The request to open the camera may include an identifier for the local camera (first camera). For example, the identifier for the phone's local camera could be "Camera 0".

[0167] 1112. Open the local camera in your phone's camera HAL.

[0168] The phone's camera HAL can open the local camera via the camera driver.

[0169] 1113. The phone's camera HAL sends a video stream captured by the local camera to the phone's instant messaging application.

[0170] The phone's camera HAL can acquire the video stream captured by the local camera through the camera driver, and then send the video stream (first video stream) captured by the local camera to the phone's instant messaging application. This allows the instant messaging application to display a video window (first video window) based on the video stream (first video stream) captured by the local camera.

[0171] 1114. The phone's camera HAL sends the orientation and / or tag attributes of the local camera to the camera mapping management module.

[0172] In one possible design, the phone's camera HAL can call a third interface to obtain the orientation attribute of the currently active local camera. This third interface could be, for example, `cameraInfo.facing`. The return value of `cameraInfo.facing` can include `CAMERA_FACING_FRONT` and `CAMERA_FACING_BACK`.

[0173] Here, CAMERA_FACING_FRONT indicates the front-facing camera attribute, and CAMERA_FACING_BACK indicates the rear-facing camera attribute. Taking camera 1 as an example, if the orientation attribute of camera 1 is CAMERA_FACING_FRONT, camera 1 is considered a front-facing camera, and its orientation attribute is marked as Front. If the orientation attribute of camera 1 is CAMERA_FACING_BACK, camera 1 is considered a rear-facing camera, and its orientation attribute is marked as Main.

[0174] The phone's camera HAL can call a fourth interface to determine the tag attributes of the currently active local camera. Taking a phone using the Qualcomm Snapdragon 8650 chip as an example, the fourth interface can include IsMainSensor, IsWideSensor, and IssteleSensor. The return values ​​of IsMainSensor, IsWideSensor, and IssteleSensor can be either true or false.

[0175] The phone's camera HAL can sequentially call interfaces such as IsMainSensor, IsWideSensor, and IssteleSensor to determine the tag attributes of the currently active local camera. The order in which the phone calls the IsMainSensor, IsWideSensor, and IssteleSensor interfaces can be a preset order or an order determined according to preset rules; this application does not impose any restrictions.

[0176] For example, IsMainSensor can be called based on the identifier (ID) of the currently active local camera. If IsMainSensor returns true, the camera is considered the main camera, and its tag attribute is marked as the main camera attribute. If IsMainSensor returns false, IsWideSensor can be called. If IsWideSensor returns true, the camera is considered a wide-angle camera, and its tag attribute is marked as wide-angle. If IsWideSensor returns false, IssteleSensor can be called. If IssteleSensor returns true, the camera is considered a telephoto camera, and its tag attribute is marked as telephoto.

[0177] 1115. The phone's camera mapping management module performs mapping and binding between the local camera and the virtual camera.

[0178] The phone's camera mapping management module can map and bind the local camera and virtual camera based on the tablet's camera information. The local camera is the phone's camera, and the virtual camera is the tablet's camera.

[0179] In this embodiment of the application, the mapping and binding of the local camera and the virtual camera can include at least the following three cases.

[0180] In the first scenario, when the tablet has only one camera (i.e., the tablet has only one camera), the local camera is mapped to this single camera, regardless of the tablet's camera attributes. This ensures that the phone can access the tablet's single camera across devices, preventing camera mapping failures that could lead to cross-device camera access failures.

[0181] It should be understood that the local camera of a mobile phone can be any local camera of the phone, such as a front camera, a rear camera, a wide-angle camera, etc., and this application does not limit it. The single camera of a tablet computer can be any type of camera, such as a front camera or a rear camera, and this application does not limit it.

[0182] In the second scenario, when the tablet has two cameras, the virtual camera that maps to the local camera is the one of the tablet's two cameras that shares the same orientation attribute as the local camera. That is, when the tablet has two cameras, we can consider both the orientation attributes of the tablet's cameras and the local camera. The local camera has a mapping relationship with the tablet's camera that shares the same orientation attribute.

[0183] It is understandable that when a tablet has two cameras, those two cameras include a front-facing camera and a rear-facing camera.

[0184] For example, if the orientation attribute of the local camera is "front-facing," meaning the local camera is a front-facing camera, then the camera with the "front-facing" orientation attribute of the tablet computer can be accessed; that is, the front-facing camera of the tablet computer can be accessed. If the orientation attribute of the local camera is "rear-facing," meaning the local camera is a rear-facing camera, then the camera with the "rear-facing" orientation attribute of the tablet computer can be accessed; that is, the rear-facing camera of the tablet computer can be accessed.

[0185] This allows the virtual camera (i.e., the tablet's camera) mapped to the phone's local camera to be determined based on its orientation attribute. This ensures that when the phone accesses the camera across devices, it can use the virtual camera with the same orientation attribute as the local camera, better meeting user needs. For example, during a video call, if the user is using the phone's local front-facing camera to capture video frames, when switching to capturing video frames using the tablet's camera, the user can use the tablet's front-facing camera, making the camera switching more aligned with the user's preference for using the front-facing camera.

[0186] In the third scenario, when a tablet has more than two cameras, the cameras that have a mapping relationship with the local camera are those among the multiple (two or more) cameras on the tablet that share the same tag attribute as the local camera. That is, when the number of cameras on the tablet is greater than two, the tag attributes of both the tablet's cameras and the local camera can be considered. The local camera has a mapping relationship with the tablet's cameras that share the same tag attribute.

[0187] For example, if the orientation attribute of the local camera is wide-angle, then the camera with the wide-angle orientation attribute on the tablet computer can be accessed; that is, the tablet computer's wide-angle camera can be accessed. If the orientation attribute of the local camera is telephoto, then the camera with the telephoto orientation attribute on the tablet computer can be accessed; that is, the tablet computer's telephoto camera can be accessed.

[0188] In this way, the virtual camera mapped from the phone's local camera (i.e., the tablet's camera) can be determined based on the tag attributes. This ensures that when the phone calls the camera across devices, it can use the virtual camera with the same tag attributes as the local camera, better meeting user needs. For example, when making a video call, if the user is using the phone's local wide-angle camera to capture video frames, when switching to capturing video frames using the tablet's camera, the user can use the tablet's wide-angle camera, making the camera switching more aligned with the user's need for a wide-angle camera.

[0189] In the third scenario, if the tablet's multiple cameras do not include a camera with the same label attributes as the local camera, the camera with a mapping relationship to the local camera is determined to be the one with the same orientation attribute as the local camera. That is, when the tablet has more than two cameras, the label attributes of the tablet's cameras and the local camera are considered first. Then, the orientation attributes of the tablet's cameras and the local camera are considered. If the virtual camera with a mapping relationship to the local camera cannot be determined based on the label attributes, the virtual camera (i.e., the tablet's camera) with a mapping relationship to the local camera is determined based on the orientation attribute.

[0190] 1116. The camera mapping management module of the mobile phone sends instruction information to the capability aggregation module of the mobile phone.

[0191] The indication information is used to indicate the camera (second camera) of the tablet computer that has a mapping relationship with the local camera. For example, the indication information may carry the identifier of the second camera.

[0192] 1117. The mobile phone's capability aggregation module sends instruction information to the tablet computer's capability acquisition module.

[0193] 1118. The tablet's capability acquisition module sends instruction information to the tablet's multimedia framework.

[0194] After receiving the instruction information, the multimedia framework of the tablet computer can collect video data (second video data) based on the camera (second camera) indicated by the instruction information.

[0195] 1119. The multimedia framework of the tablet computer sends the video data captured by the tablet computer's camera to the camera mapping management module of the mobile phone.

[0196] The tablet's multimedia framework can send video data (video frames / streams) captured by the tablet's camera to the mobile phone via the tablet's transmission channel module. The mobile phone's camera mapping management module can receive the video data sent by the tablet via the mobile phone's transmission channel module.

[0197] 1120. The phone's camera mapping management module sends video data captured by the tablet's camera to the phone's camera HAL.

[0198] 1121. The phone's camera HAL sends video data captured by the tablet's camera to an instant messaging application.

[0199] The mobile phone's instant messaging application displays a second video window based on the video data (second video data) captured by the tablet's camera.

[0200] In other words, the mobile phone's instant messaging application can replace the video data captured by the local camera with the video data captured by the tablet's camera, thereby achieving camera switching.

[0201] In some embodiments, when multi-screen collaboration is established between a mobile phone and a tablet, after opening the camera app on the mobile phone, the user can select either the local camera or the tablet's camera via the "More" - "Multi-camera" path. For the mobile phone, the tablet's camera can be considered a virtual camera. Figure 12 As shown, in response to a user's action of selecting a camera in the phone's camera application, the phone can display a pop-up window 12, which may include multiple camera options. These multiple camera options may include options corresponding to the local camera and options corresponding to the tablet's camera. The local camera may include the local front-facing camera (i.e., native (front-facing)), the local rear main camera (i.e., native (rear-facing)), and the local wide-angle camera (i.e., native (wide-angle)). The tablet's camera may include the tablet's front-facing camera (i.e., my tablet (front-facing)), the tablet's rear main camera (i.e., my tablet (rear-facing)), and the tablet's wide-angle camera (i.e., my tablet (wide-angle)). Optionally, the local camera may also include a local telephoto camera, macro camera, etc. The tablet's camera may also include the tablet's telephoto camera, macro camera, etc. Thus, in a virtualized camera scenario (i.e., a scenario where an electronic device (e.g., a phone) calls the camera across devices), the phone can call multiple types of cameras (e.g., front-facing camera, rear main camera, wide-angle camera, etc.) across devices. Mobile phones can offer users a variety of camera types to choose from, which can enrich user options and improve the user experience.

[0202] Based on the method provided in this application, in a virtualized camera scenario (i.e., a scenario where a first electronic device (e.g., a mobile phone) calls the camera of a second electronic device (e.g., a tablet computer) across devices), the mobile phone can consider information such as the number of cameras, orientation attributes, and marking attributes of the tablet computer's cameras when calling the camera across devices, so that the mobile phone and the tablet computer can perform camera collaboration more accurately.

[0203] For example, a mobile phone can determine the virtual camera mapped from its local camera (i.e., the tablet's camera) based on its orientation attribute. This ensures that when the phone calls a camera across devices, it can use the virtual camera with the same orientation attribute as the local camera, better meeting user needs. For instance, during a video call, if the user is using the phone's local front-facing camera to capture video frames, when switching to capturing video frames using the tablet's camera, the user can use the tablet's front-facing camera, making the camera switching more aligned with the user's preference for using the front-facing camera.

[0204] For example, a mobile phone can determine the virtual camera (i.e., the tablet's camera) mapped to its local camera based on the tag attributes. This ensures that when the phone accesses the camera across devices, it can use the virtual camera with the same tag attributes as the local camera, better meeting user needs. For instance, during a video call, if the user is using the phone's local wide-angle camera to capture video frames, when switching to capturing video frames using the tablet's camera, the user can use the tablet's wide-angle camera, making the camera switching more aligned with the user's need for a wide-angle camera.

[0205] Some embodiments of this application provide an electronic device that may include a touchscreen, a memory, and one or more processors. The touchscreen, memory, and processors are coupled. The memory stores computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the electronic device in the above method embodiments. The structure of the electronic device can be referred to... Figure 2 The structure of the electronic device 100 shown.

[0206] This application also provides a chip system (e.g., a system-on-a-chip (SoC)). Figure 13 As shown, the chip system includes at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 are interconnected via lines. For example, the interface circuit 1302 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 1302 can be used to send signals to other devices (e.g., the processor 1301 or the touchscreen of an electronic device). Exemplarily, the interface circuit 1302 can read instructions stored in the memory and send those instructions to the processor 1301. When the instructions are executed by the processor 1301, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0207] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0210] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0212] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0213] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cross-device camera collaboration method, characterized in that, Applied to a first electronic device, wherein the first electronic device and a second electronic device are in a multi-screen collaborative state, the method includes: In response to a user triggering a video call, the first electronic device acquires first video data based on its first camera and displays a first video window based on the first video data. The first electronic device acquires camera information from the second electronic device; wherein, the camera information includes the number of cameras of the second electronic device, the camera identifier and camera attributes, the camera attributes include orientation attributes, the orientation attributes include front-facing attributes or rear-facing attributes, the camera attributes also include tag attributes, the tag attributes include at least one of wide-angle attributes, main camera attributes, telephoto attributes or macro attributes; The first electronic device acquires the orientation attribute and / or label attribute of the first camera; The first electronic device determines a second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device, as well as the orientation attribute and / or label attribute of the first camera; When the number of cameras in the second electronic device is 1, the second camera is the only camera in the second electronic device; When the number of cameras in the second electronic device is 2, the second camera is the camera among the two cameras of the second electronic device that has the same orientation attribute as the first camera; When the number of cameras of the second electronic device is greater than 2, the second camera is the camera among the multiple cameras of the second electronic device that has the same tag attribute as the first camera; The first electronic device sends indication information to the second electronic device, the indication information being used to instruct the second camera; The first electronic device receives second video data from the second electronic device, the second video data being derived from the second camera; The first electronic device displays the second video window based on the second video data.

2. The method according to claim 1, characterized in that, The method further includes: If the plurality of cameras of the second electronic device does not include a camera with the same tag attribute as the first camera, then the second camera that has a mapping relationship with the first camera is determined to be the camera among the plurality of cameras of the second electronic device that has the same orientation attribute as the first camera.

3. A cross-device camera collaboration method, characterized in that, A method applicable to a system comprising a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device are in a multi-screen collaborative state, the method comprising: In response to a user triggering a video call, the first electronic device acquires first video data based on its first camera and displays a first video window based on the first video data. The first electronic device sends a request message to the second electronic device, the request message being used to request camera information from the second electronic device; The second electronic device receives the request message and collects camera information of the second electronic device. The camera information includes the number of cameras of the second electronic device, the camera identification and camera attributes. The camera attributes include orientation attributes, which include front-facing attributes or rear-facing attributes. The camera attributes also include tag attributes, which include at least one of wide-angle attributes, main camera attributes, telephoto attributes or macro attributes. The second electronic device sends the camera information of the second electronic device to the first electronic device; The first electronic device receives camera information from the second electronic device. The first electronic device acquires the orientation attribute and / or label attribute of the first camera; The first electronic device determines a second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device, as well as the orientation attribute and / or label attribute of the first camera; When the number of cameras in the second electronic device is 1, the second camera is the only camera in the second electronic device; When the number of cameras in the second electronic device is 2, the second camera is the camera among the two cameras of the second electronic device that has the same orientation attribute as the first camera; When the number of cameras of the second electronic device is greater than 2, the second camera is the camera among the multiple cameras of the second electronic device that has the same tag attribute as the first camera; The first electronic device sends indication information to the second electronic device, the indication information being used to instruct the second camera; The second electronic device receives the instruction information and, based on the instruction information, acquires second video data using the second camera; The second electronic device sends the second video data to the first electronic device; The first electronic device receives the second video data from the second electronic device; The first electronic device displays the second video window based on the second video data.

4. The method according to claim 3, characterized in that, The method further includes: If the plurality of cameras of the second electronic device does not include a camera with the same tag attribute as the first camera, then the second camera that has a mapping relationship with the first camera is determined to be the camera among the plurality of cameras of the second electronic device that has the same orientation attribute as the first camera.

5. A system for cross-device camera collaboration, characterized in that, The system includes a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device are in a multi-screen collaborative state, wherein: The first electronic device is configured to, in response to a user's operation of triggering a video call, acquire first video data based on the first camera of the first electronic device, and display a first video window based on the first video data; The first electronic device is further configured to send a request message to the second electronic device, the request message being used to request camera information from the second electronic device; The second electronic device is used to receive the request message and collect camera information of the second electronic device. The camera information includes the number of cameras of the second electronic device, the camera identification and camera attributes. The camera attributes include orientation attributes, which include front-facing attributes or rear-facing attributes. The camera attributes also include tag attributes, which include at least one of wide-angle attributes, main camera attributes, telephoto attributes or macro attributes. The second electronic device is also used to send camera information of the second electronic device to the first electronic device; The first electronic device is further configured to receive camera information from the second electronic device. The first electronic device is further configured to acquire the orientation attribute and / or label attribute of the first camera; The first electronic device is further configured to determine a second camera that has a mapping relationship with the first camera based on the camera information of the second electronic device and the orientation attribute and / or tag attribute of the first camera; when the number of cameras of the second electronic device is 1, the second camera is the only camera of the second electronic device; when the number of cameras of the second electronic device is 2, the second camera is the camera with the same orientation attribute as the first camera among the two cameras of the second electronic device; when the number of cameras of the second electronic device is greater than 2, the second camera is the camera with the same tag attribute as the first camera among the multiple cameras of the second electronic device. The first electronic device is further configured to send indication information to the second electronic device, the indication information being used to instruct the second camera; The second electronic device is further configured to receive the instruction information and, based on the instruction information, acquire second video data using the second camera; The second electronic device is further configured to send the second video data to the first electronic device; The first electronic device is further configured to receive second video data from the second electronic device; The first electronic device is also used to display a second video window based on the second video data.

6. An electronic device, characterized in that, The electronic device includes: a wireless communication module, a memory, and one or more processors; the wireless communication module, the memory, and the processor are coupled together. The memory is used to store computer program code, which includes computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method as described in claim 1 or 2.

7. A computer-readable storage medium, characterized in that, Includes computer instructions; When the computer instructions are executed on the electronic device, the electronic device causes the electronic device to perform the method as described in claim 1 or 2.

Images