Camera switching method and electronic device
By creating a second session between the first device and the second device to keep the link alive, the problem of large camera switching delay when calling the camera across devices is solved, and a faster camera switching process is achieved.
Patent Information
- Application Number
- CN202311267256.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-09-26
AI Technical Summary
When calling the camera across devices, there is a large delay during the camera switching process, mainly due to the steps of link disconnection and reconstruction, the delay increases.
Keep the first link alive by creating a second session between the first device and the second device, avoiding disconnecting the link immediately after closing the first camera until the third session is successfully established, thus omitting Link disconnection and reconstruction steps.
It effectively reduces the latency of camera switching and improves the flexibility and response speed of calling cameras across devices.
Smart Images

Figure CN118450249B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a camera switching method and an electronic device. Background Art
[0002] When using some applications in electronic devices such as tablets and laptops, you may need to call the camera to take pictures or videos at any time. For example, when holding a video conference through a laptop, you may need to use the laptop's built-in camera to take and display videos while the laptop displays the video conference document. However, due to the large size of the laptop or the limited camera capability of the built-in camera, the flexibility of shooting with the built-in camera is not high.
[0003] In order to improve the flexibility of shooting, it is currently possible to shoot by calling the camera across devices. For example, when holding a video conference through a laptop, the laptop calls the camera of the mobile phone to shoot, and displays the picture taken by the mobile phone camera. However, when calling the camera across devices, if it involves switching cameras, such as switching the front camera of the called mobile phone to the rear camera of the mobile phone, it is necessary to disconnect the link between the laptop and the mobile phone when turning off the front camera of the mobile phone, and then re-establish a new link and turn on the rear camera of the mobile phone. In this process, the link disconnection and reconstruction process will cause a large delay in camera switching. How to reduce the delay of camera switching when calling the camera across devices has become one of the urgent problems to be solved. Summary of the invention
[0004] The present application provides a camera switching method and an electronic device, which can reduce the delay of camera switching when calling cameras across devices.
[0005] In a first aspect, the present application provides a camera switching method, which is applied to a first device, and the method includes: displaying a first control when the first device receives a first video stream captured by a first camera of a second device; the first device establishes a first link with the second device; the first video stream is transmitted via a first session carried in the first link; the first control is used to switch the camera called by the first device to the second camera of the second device; in response to the user's operation on the first control, a second session is established with the second device; the second session is carried on the first link and is used to keep the first link alive; a first close request is sent to the second device; the first close request is used to instruct the second device to close the first session and close the first camera; after the first session is successfully closed, a third session is established with the second device, and a first open request is sent to the second device; the first open request is used to instruct the second device to start the second camera; the third session is carried on the first link; receiving a second video stream; the second video stream is captured via the second camera of the second device, and transmitted to the first device via the third session.
[0006] In summary, for the scenario where the first device switches the called camera from the first camera of the second device to the second camera, the present application can create a second session between the first device and the second device to keep the first link alive before sending the first shutdown request to the second device. Based on this, even after the second device turns off the first camera and disconnects the first session with the first device, the first link will not be disconnected. Accordingly, there is no need to create a new link to carry the third session before the second device turns on the second camera and creates the third session. It can be seen that the present application can reduce the delay of camera switching by omitting the disconnection and reconstruction steps of the first link.
[0007] In a possible implementation, the keep-alive duration of the second session is greater than or equal to a time interval between a time when the second session is successfully established and a time when the third session is successfully established.
[0008] For the first link, once a session is carried on the first link, the first link will not be disconnected. Therefore, when creating the second session, the keep-alive duration of the second session can be set, and by setting the keep-alive duration of the second session to last until the third session is successfully established, the second session can fully ensure that the first link will not be disconnected before the third session is successfully established, that is, there is no need to rebuild the first link.
[0009] In a possible implementation, the second device is a device within the near field communication range of the first device and / or a device logged in with the same user account as the first device.
[0010] In this way, near-field communication can be used between mutually trusted devices to call cameras and transmit video streams across devices.
[0011] In a possible implementation, the first control includes a camera flip control, and the camera flip control is used to implement flip switching between the first camera and the second camera; or, the first control includes an option control for the second camera.
[0012] That is, the first device may display a camera flip control, and the user may directly click the camera flip control to switch the camera called by the first device from the first camera to the second camera. Alternatively, the first device displays option controls for multiple cameras, and the option controls for multiple cameras include option controls for the second camera, then the user may directly click the option control for the second camera (equivalent to the user selecting the second camera) to switch the camera called by the first device from the first camera to the second camera.
[0013] In a second aspect, the present application provides a camera switching method, which is applied to a second device, and the method includes: establishing a second session with the first device when sending a first video stream captured by a first camera of the second device to the first device; the first device and the second device establish a first link; the first video stream is transmitted via the first session carried in the first link; the second session is carried on the first link and is used to keep the first link alive; receiving a first close request sent by the first device, and closing the first session and the first camera in response to the first close request; establishing a third session with the second device after the first session is successfully closed; receiving a first open request sent by the first device, and starting the second camera in response to the first open request; sending a second video stream to the first device; the second video stream is captured via the second camera and transmitted to the first device via the third session.
[0014] In summary, for the scenario where the first device switches the called camera from the first camera of the second device to the second camera, the present application can create a second session between the first device and the second device to keep the first link alive before the second device successfully turns off the first camera. Based on this, even after the second device turns off the first camera and disconnects the first session with the first device, the first link will not be disconnected. Accordingly, there is no need to create a new link to carry the third session before the second device turns on the second camera and creates the third session. It can be seen that the present application can reduce the delay of camera switching by omitting the disconnection and reconstruction steps of the first link.
[0015] In a possible implementation, the keep-alive duration of the second session is greater than or equal to a time interval between a time when the second session is successfully established and a time when the third session is successfully established.
[0016] In this way, the keep-alive duration of the second session can be set to last until the third session is successfully established, so that the second session can fully ensure that the first link will not be disconnected before the third session is successfully established, that is, there is no need to rebuild the first link.
[0017] In a possible implementation, the first device includes a camera management service, and the second device includes a camera business service and a camera management service; the above-mentioned establishment of a second session with the first device includes: the camera management service of the second device establishes a second session with the camera management service of the first device; the above-mentioned closing of the first session and the first camera includes: after the camera management service of the second device establishes the second session with the camera management service of the first device, the camera management service of the second device detects whether a first instruction is received; the first instruction is used to instruct to close the first session and the first camera; if the camera management service of the second device receives the first instruction, the camera management service of the second device closes the first session and the first camera; or, if the camera management service of the second device does not receive the first instruction, the camera management service of the second device queries the camera business service for the service status of the first camera; if the service status of the first camera is a no service status, the camera management service of the second device closes the first session and the first camera.
[0018] In this way, after the camera management service of the second device establishes the second session with the camera management service of the first device, the camera management service of the second device can execute the closing of the first session and the first camera by detecting whether the first instruction is received. At the same time, in the case where the first instruction is not received, since the camera management service may not receive the first instruction due to transmission abnormality or transmission delay during the process of sending the first instruction from the camera business service to the camera management service, the camera management service can actively query the service status of the first camera from the camera business service, and execute the closing of the first session and the first camera according to the service status of the first camera. Based on this, when the first instruction is not received, by actively querying the service status of the first camera, the working status (such as start / stop) of the first camera can be kept consistent with the service status.
[0019] In a possible implementation, before closing the first session and the first camera, the method further includes: in response to a first closing request sent by the first device, the camera business service of the second device updates the service state of the first camera from a service state to a no service state; and the camera business service of the second device sends a first instruction to the camera management service of the second device.
[0020] In this way, when the second device receives the first shutdown request, it can update the service status of the first camera to a no service status in the camera business service of the second device, and send a first instruction to the camera management service of the second device, so that the camera management service of the second device can decide whether to shut down the first camera and the first session based on the first instruction or the service status of the first camera.
[0021] In one possible implementation, the above-mentioned starting the second camera in response to the first opening request sent by the first device includes: in response to the first opening request sent by the first device, the camera business service of the second device sends a second instruction to the camera management service of the second device; the second instruction is used to instruct to start the second camera; the camera management service of the second device queries the working status of the first camera and the working status of the second camera in response to the second instruction; if the working status of the first camera is the starting state and the working status of the second camera is the off state, then the first camera is turned off, and after the first camera is successfully turned off, the second camera is started; or, if the working status of the first camera is the off state and the working status of the second camera is the off state, then the second camera is started.
[0022] Since the camera management service of the second device has not completed the closing process of the first camera when the first opening request is received, it is necessary to query the working status of the first camera and the second camera before opening the second camera to ensure that no errors or abnormalities occur when opening the second camera, thereby improving the accuracy of camera switching.
[0023] In a possible implementation, the second device is a device within the near field communication range of the first device and / or a device logged in with the same user account as the first device.
[0024] In this way, near-field communication can be used between mutually trusted devices to call cameras and transmit video streams across devices.
[0025] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the camera switching method in the first aspect and any possible implementation thereof, or executes the camera switching method in the second aspect and any possible implementation thereof.
[0026] In a fourth aspect, the present application provides a communication system, which includes a first device and a second device in any possible implementation of the first aspect or the second aspect, and is used to execute the camera switching method in the first aspect, the second aspect and any possible implementation of the first aspect, the second aspect and any possible implementation of the second aspect.
[0027] In a fifth aspect, the present application provides a camera switching device, which includes a function / unit for executing the camera switching method in the above-mentioned first aspect and any possible implementation thereof, or a function / unit for executing the camera switching method in the above-mentioned second aspect and any possible implementation thereof.
[0028] In a sixth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes at least one processor and an interface, and the interface is used to receive computer instructions and transmit them to at least one processor; the at least one processor runs the computer instructions to enable the electronic device to execute the camera switching method in the above-mentioned first aspect and any possible implementation thereof, or execute the camera switching method in the above-mentioned second aspect and any possible implementation thereof.
[0029] In the seventh aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the camera switching method in the above-mentioned first aspect and any possible implementation thereof, or executes the camera switching method in the above-mentioned second aspect and any possible implementation thereof.
[0030] In an eighth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the camera switching method in the above-mentioned first aspect and any possible implementation thereof, or execute the camera switching method in the above-mentioned second aspect and any possible implementation thereof.
[0031] It can be understood that the beneficial effects that can be achieved by the electronic device, communication system, camera switching device, chip system, computer-readable storage medium, and computer program product provided above can refer to the beneficial effects in the first aspect, the second aspect and any possible implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A schematic diagram of the composition of a communication system provided in an embodiment of the present application;
[0033] Figure 2A A schematic diagram of a camera switching scenario provided in an embodiment of the present application;
[0034] Figure 2B A schematic diagram of another camera switching scenario provided in an embodiment of the present application;
[0035] Figure 3 A hardware structure diagram of an electronic device provided in an embodiment of the present application;
[0036] Figure 4AA software architecture diagram of a first device provided in an embodiment of the present application;
[0037] Figure 4B A software architecture diagram of a second device provided in an embodiment of the present application;
[0038] Figure 5 A schematic diagram of a camera switching method according to an embodiment of the present invention;
[0039] Figure 6 A schematic diagram of a scene showing a first control provided in an embodiment of the present application;
[0040] Figure 7 A software interaction flow chart of a camera switching method provided in an embodiment of the present application;
[0041] Figure 8 A structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.
[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0044] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that the user can recognize. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as time, date, text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, widget, etc. displayed on the display screen of an electronic device.
[0045] For ease of understanding, the communication system in the embodiment of the present application is first introduced below:
[0046] The present application embodiment provides a communication system, see Figure 1 , the communication system includes a first device (such as Figure 1 The laptop computer 110 shown) and at least one second device (such as Figure 1 The mobile phone 120 and the tablet 130 are shown).
[0047] Among them, the user can call the camera of any second device to take pictures through the first device, and any second device returns the image data (such as video stream) collected by the called camera to the first device, and the first device displays based on the received image data.
[0048] When a user uses the first device to call the camera of any second device to shoot, it includes opening the camera, closing the camera, and switching the camera. Take the case where the laptop 110 calls the camera of the tablet 130 as an example:
[0049] Turn on the camera: When the user needs to turn on any camera of the tablet 130 through the laptop 110, the laptop 110 can send a camera turn-on request to the tablet 130; after receiving the turn-on request, the tablet 130 turns on the corresponding camera to shoot in response to the turn-on request. Then the tablet 130 can send the image data collected by the turned-on camera to the laptop 110 for display.
[0050] Turn off the camera: When the user needs to turn off any camera of the tablet 130 through the laptop computer 110, the laptop computer 110 can send a camera turn-off request to the tablet 130. After receiving the turn-off request, the tablet 130 turns off the corresponding camera in response to the turn-off request and stops sending image data collected by the camera.
[0051] Switching cameras: When the user needs to switch the camera of the tablet 130 that has been called through the laptop computer 110, the laptop computer 110 can send a shutdown request for the first camera (such as the front camera) and a startup request for the second camera (such as the rear camera) to the tablet 130. After receiving the shutdown request for the first camera and the startup request for the second camera, the tablet 130 turns off the first camera and starts the second camera. Then the tablet 130 can send the image data collected by the started second camera to the laptop computer 110 for display. Based on this, the laptop computer 110 can switch the camera called across devices (such as switching the called camera from the front camera of the tablet 130 to the rear camera of the tablet 130).
[0052] Before the first device calls the camera of any second device to shoot, a communication connection can be established between the first device and any second device, and the communication connection can be a near field communication connection, wherein near field communication includes Bluetooth, Wireless Fidelity (Wireless Fidelity, Wi-Fi), Near Field Communication (Near Field Communication, NFC), etc.
[0053] Through the communication connection established between the first device and any second device, the first device can send the above-mentioned related requests (such as closing requests and opening requests) to any second device based on the control link in a point-to-point manner, so that any second device turns on or off the camera. In addition, any second device can send image data (such as video stream) collected by the camera to the first device based on the data link in a point-to-point manner. It is understandable that the data link can also be used to transmit other data, such as text data, audio data, etc.
[0054] For the data link between the first device and any second device, different types of sessions can be pre-created on the data link to transmit different types of data. For example, session A is of the video type, used to transmit video streams; session B is of the bytes type, used to transmit audio data, etc.; session C is of the HTML type, used to transmit text data. When no session exists in the data link between the first device and any second device, the data link is disconnected.
[0055] Regarding the above-mentioned case of switching cameras: when the tablet 130 turns off the first camera, the session for transmitting the image data collected by the first camera will also be closed. In this case, since the session for transmitting the image data collected by the first camera is closed, the data link between the laptop 110 and the tablet 130 is disconnected. Therefore, before the tablet 130 turns on the second camera, it is necessary to re-establish the data link between the laptop 110 and the tablet 130, and then create a session for transmitting the image data collected by the second camera, and turn on the second camera. It can be seen that the disconnection and reconstruction of the data link involved in this process will increase the delay of camera switching.
[0056] Exemplarily, the first device / second device may be a handheld device, a vehicle-mounted device, etc., such as a mobile phone, a tablet, a laptop computer, a PDA, a mobile internet device (MID), a virtual reality (VR) device, an augmented reality (AR) device, a wireless device in industrial control, a wireless device in self-driving, a wireless device in remote medical surgery, a wireless device in a smart grid, a wireless device in transportation safety, a wireless device in a smart city, a wireless device in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a personal computer (PC), a wireless communication device, ... Computer, PC), computing device or other processing device connected to a wireless modem, vehicle-mounted device, wearable device, terminal device in a 5G network, or terminal device in a future evolved public land mobile communication network (public landmobile network, PLMN), etc., the embodiments of the present application are not limited to this.
[0057] The first device / second device may also be a wearable device, such as a smart watch or smart glasses, and may focus on a certain type of application function and be used in conjunction with other devices such as smart phones, such as various smart bracelets and smart jewelry for vital sign monitoring.
[0058] The first device / second device mentioned above may also be a terminal device in an Internet of Things (IoT) system.
[0059] The present application proposes a camera switching method and an electronic device, which can reduce the delay of camera switching when calling cameras across devices.
[0060] The method proposed in this application can be applied to the switching scenario of calling the camera across devices. The following is an example of the camera switching scenario when calling the camera across devices:
[0061] In one example, see Figure 2A When the laptop computer 110 uses the front camera of the tablet computer 130 to take a photo, the laptop computer 110 may display Figure 2A The interface 20 shown in FIG. 1 includes a captured image (such as a portrait), camera call information (such as “the front camera of the tablet 130 is being called”), a flip control 210, a pause control, and a disconnect control. The tablet 130 displays the following: Figure 2A The interface 21 shown in the figure includes camera usage information (such as "front camera in use"), device connection information (such as "connected to notebook 110"), pause control, and disconnection control. The notebook computer 110 can receive a user's click operation on the flip control 210 to switch the camera of the tablet 130 called by the notebook computer 110 from the front camera to the rear camera. For example, after the switch is completed, the notebook computer 110 displays Figure 2A The interface 22 shown in FIG. 1 includes the captured image (such as a puppy), camera call information (such as "the rear camera of the tablet 130 is being called"), a flip control 210, a pause control, and a disconnect control. The tablet 130 displays the following: Figure 2A As shown in interface 23, interface 21 includes camera usage information (such as "rear camera in use"), device connection information (such as "connected to notebook 110"), pause control and disconnect control.
[0062] In another example, see Figure 2A When the laptop 110 uses the front camera of the tablet 130 to take a picture, the user can view the list of available cameras of the laptop 110 on the laptop 110. For example, before the laptop is switched, the following is displayed: Figure 2BThe interface 24 shown in the figure includes a list of selectable cameras, including a selection control 220 for the front camera of the laptop computer 110, the rear camera of the laptop computer 110, the front camera of the mobile phone 120, the rear camera of the mobile phone 120, the front camera of the tablet 130, and the rear camera of the tablet 130. The selection control for the front camera of the tablet 130 is also used to prompt the user that the front camera of the tablet 130 is being called. The user can click the selection control 220 for the rear camera of the tablet 130 to switch the camera called by the laptop computer 110 to the rear camera of the tablet 130. After the switch is completed, the laptop computer 110 can display the following Figure 2B The interface 25 shown in the figure still includes the selection controls of the above-mentioned cameras, and the selection control 220 of the rear camera of the tablet 130 is used to prompt the user that the rear camera of the tablet 130 is being called. In this example, the interface 21 and the interface 23 displayed by the tablet 130 can refer to Figure 2A The corresponding description.
[0063] It is understandable that in addition to the above two examples, users can also trigger the scene of switching cameras across devices through other methods, and this application does not limit this.
[0064] Moreover, the present application can be applied to video conferencing, video teaching, inserting pictures, etc. For example, the present application can call the camera across devices when using remote conferencing applications, live broadcast applications, remote teaching applications, etc.
[0065] The electronic device proposed in this application includes a first device and a second device. The relationship between the first device and the second device can be seen in the above Figure 1 As described in the present application, the method proposed in this application can be applied to Figure 1 The hardware architecture and software architecture of the first device / the second device are introduced below:
[0066] See also Figure 3 , is a hardware structure diagram of an electronic device (such as a first device / a second device) provided in an embodiment of the present application. Figure 3As shown, taking the electronic device as a mobile phone as an example, it may include: a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.
[0067] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0068] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0069] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.
[0070] The charging management module 340 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While the charging management module 340 charges the battery 342, it can also power the electronic device through the power management module 341.
[0071] The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, and the wireless communication module 360. The power management module 341 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc.
[0072] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.
[0073] The electronic device implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute program instructions to generate or change display information.
[0074] The electronic device can realize the shooting function through ISP, camera 393, video codec, GPU, display screen 394 and application processor.
[0075] In some embodiments, when the electronic device acts as a second device, it can call the camera 393 to capture images based on a request from the first device and return the images to the first device.
[0076] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.
[0077] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0078] The electronic device can implement audio functions such as music playing and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.
[0079] The key 390 includes a power key, a volume key, etc. The key 390 can be a mechanical key or a touch key. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.
[0080] Motor 391 can generate vibration prompts. Motor 391 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 394, motor 391 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
[0081] Indicator 392 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0082] The SIM card interface 395 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device by inserting the SIM card interface 395 or pulling the SIM card out of the SIM card interface 395. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.
[0083] In the embodiment of the present application, the software system of the electronic device (such as the first device / the second device) can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structures of the first device and the second device are exemplified.
[0084] See also Figure 4A , is a software architecture diagram of the first device provided in the embodiment of the present application. Figure 4B , is a software architecture diagram of the second device provided in the embodiment of the present application. Figure 4A and Figure 4B As shown in FIG. 1 , the layered architecture divides the software into several layers, each with a clear role and division of labor. Data is transmitted and communicated between layers through software interfaces. In some embodiments, The system is divided into four layers, from top to bottom: application layer, application framework layer, hardware abstract layer (HAL), and kernel layer.
[0085] ①. The application layer can include a series of application packages. Figure 4A As shown, the application layer of the first device includes third-party applications and camera business services. Figure 4B As shown, the application layer of the second device includes a camera business service.
[0086] Among them, the third-party application refers to an application that can call the camera of a trusted device. For example, the third-party application includes but is not limited to a remote conference application, a live broadcast application, a remote teaching application, etc. The camera business service is used to provide the relevant capabilities of calling the camera across devices, such as the discovery, enabling, starting / ending the camera service of the trusted device, and updating the service status of the camera (such as no service status / service status). In this application, the camera business service of the first device is used to receive the relevant instructions sent by the camera management service of the first device, so as to notify the camera business service of the second device to manage the camera service provided by the second device.
[0087] ②. The application framework layer provides application developers with an application programming interface (API) framework and various services and management tools for accessing core functions. The application framework layers of the first device and the second device both include a camera management service, which may be a virtualization platform (distributed mobile sensing development platform, DMSDP) for providing virtual (such as cross-device calls) camera capabilities, including: turning off / starting the camera, creating or closing a session for carrying image data collected by the camera. In the present application, the camera management service of the first device is used to send relevant instructions to the camera business service of the first device, thereby notifying the camera business service of the second device to manage the camera service provided by the second device; and the camera management service of the first device is used to manage the relevant sessions between the first device and the second device. The camera management service of the second device is used to receive relevant instructions sent by the camera business service of the second device, thereby turning off / on the camera and closing the session.
[0088] ③. The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. The hardware abstraction layer of the first device includes communication hardware abstraction and display hardware abstraction; the hardware abstraction layer of the second device includes communication hardware abstraction, display hardware abstraction, and camera hardware abstraction.
[0089] Communication hardware abstraction is used to implement data transmission across devices, device self-discovery, etc. For example, through communication hardware abstraction, a first device can discover a second device located in the near field and establish a data link and a control link with the second device. Then, image data can be transmitted on the data link, and related requests can be transmitted on the control link.
[0090] The display hardware abstraction is used to implement the display function of the device and control the display area, etc. For example, the first device can display the screen corresponding to the received image data and display the scheduling information of the camera through the display hardware abstraction.
[0091] The camera hardware abstraction is used to control the camera startup / shutdown in response to the camera management service.
[0092] ④. The kernel layer is the layer between hardware and software. The kernel layers of the first device and the second device may include display drivers and communication drivers, etc. Communication drivers include but are not limited to Bluetooth drivers, Wi-Fi drivers, etc. The kernel layer of the second device may also include a camera driver. These individual drivers are used to respond to the corresponding hardware abstract instructions in the hardware abstraction layer to achieve the corresponding functions. For example, the camera driver in the second device is used to respond to the camera hardware abstraction layer to control the camera to start / stop.
[0093] It can be understood that the above software architecture is only an example. In specific implementation, the first device / second device may further include more functional modules in the above layers, which will not be elaborated in this application.
[0094] The camera switching method provided in the embodiment of the present application can be implemented in a communication system composed of a first device and a second device having the above-mentioned software and hardware structure. The image capture method provided in the embodiment of the present application will be further described in detail below in conjunction with the accompanying drawings.
[0095] See also Figure 5 The camera switching method provided in the embodiment of the present application includes S501 to S513, wherein S501 to S505 are steps performed before the camera is switched when the camera is called across devices; S506 to S513 are steps performed when the camera is switched. The following first introduces the steps performed before the camera is switched:
[0096] S501: A first device obtains an operation of opening a first camera.
[0097] In the embodiment of the present application, a communication connection is established between the first device and the second device, and the first camera is a camera of the second device. Figure 1 In the example, the laptop computer 110 is used, the second device is a tablet computer 130 , and the tablet computer 130 includes a front camera and a rear camera. Then, the first camera is the front camera or the rear camera of the tablet computer 130 .
[0098] Optionally, the first device may obtain the user's operation of opening the first camera when running the target application. The first camera may be the camera that was called when the second device was called across devices last time. Alternatively, the first camera is the camera manually selected by the user when calling the camera across devices this time. For example, the target application is any of the third-party applications such as remote conferencing applications, live broadcast applications, and remote teaching applications. Taking the target application as a remote conferencing application as an example, when the user uses the remote conferencing application to access a meeting, the user can choose whether to start the camera. If the user chooses to start the camera, and the user does not specify which camera to start, the first device can call the camera that was called across devices last time. The device corresponding to the last cross-device call is also the second device, and the camera that was called across devices last time is also the first camera. After the user chooses to start the camera, the first device displays a list of optional cameras (the list of optional cameras can be referred to Figure 2B In interface 25), the user initiates a selection operation on a camera of the second device in the optional camera list, and the camera of the second device selected by the user is the first camera.
[0099] Optionally, before the first device obtains the operation of opening the first camera, the first device needs to determine that the second device is an available device that supports remote calling of the camera and can transmit image data using near field communication.
[0100] The first device can determine that the second device can support remote camera invocation and recording by receiving device synchronization information. The device synchronization information can be sent when the first device and the second device are pre-connected. Alternatively, if the second device and the first device log in to the same user account, the second device can synchronize its ability to remotely invoke the camera to the first device.
[0101] At the same time, before the first device obtains the operation of turning on the first camera, the first device needs to scan whether the recorded second device that supports remote camera calls is within the near field communication range to determine whether the first device and the second device can perform near field communication. If both the first device and the second device turn on Bluetooth, or the first device and the second device are connected to the same Wi-Fi, etc., it can be determined that the first device and the second device can perform near field communication, and accordingly, the second device is an available device (or online device) for this cross-device camera call. It should be noted that the first device can display the second device in the list of available cameras or determine that the first camera is a camera of the second device in the last cross-device call when receiving the operation of turning on the first camera when it determines that the second device is an available device for this cross-device camera call.
[0102] S502: The first device establishes a first session with the second device, where the first session is carried on a first link.
[0103] Among them, the first link is a data link established between the first device and the second device, and the data link can be a P2P link. The first device can establish the first link with the first device after determining that the second device is an available device that supports remote calling of the camera and can use near-field communication to transmit image data.
[0104] The first session is used to transmit image data collected by the first camera, such as the first video stream. In the present application, the first device may send a negotiation message to the second device, and accordingly, the second device receives the negotiation message and establishes a first session with the first device. The negotiation message includes a device identifier, a service type of the session, and service parameters of the session. Among them, the device identifier is used to indicate the peer device of the first device in the first session, the service type is used to indicate the type of data used to transmit the first session (including video type, bytes type, etc.), and the service parameters include the keep-alive duration (or duration) of the first session, etc. For example, the device identifier is the identifier of the second device, the service type is the video type, and the service parameters include that the keep-alive duration of the first session is not set, or is set to the maximum value that a video type session can survive, etc., to ensure the reliable transmission of the first video stream.
[0105] S503: The first device sends a request to open the first camera to the second device.
[0106] Accordingly, the second device may receive a request to open the first camera.
[0107] In the present application, the first device may send a request to open the first camera to the second device during the process of establishing the first session with the second device. That is, the first device may execute the above S502 and S503 at the same time. The request to open the first camera includes the identifier of the first device and the identifier of the first camera, and the request to open the first camera is sent on the control link between the first device and the second device, and the control link is not the same link as the first link.
[0108] S504: The second device starts the first camera in response to the opening request of the first camera.
[0109] After receiving the opening request of the first camera, the second device may start the first camera.
[0110] Optionally, after receiving the opening request of the first camera, the second device may first obtain user authorization, and then start the first camera after obtaining the user authorization.
[0111] S505: The second device sends a first video stream captured by the first camera to the first device, and the first video stream is transmitted in the first session.
[0112] After the first session is successfully created and the first camera is successfully started, the first camera of the second device starts to capture the first video stream and sends the captured first video stream to the first device via the first session. Accordingly, the first device can receive the first video stream and display it according to the first video stream.
[0113] Optionally, the second device may display a prompt interface for camera use, for example, the second device displays Figure 2A or Figure 2B Interface 21 in the figure is used to prompt the user that the second device is using the second camera (such as the front camera) to take pictures.
[0114] Among them, the above S501 to S505 are steps performed by the first device and the second device before the camera is switched when the camera is called across devices. The following describes the steps performed during the camera switching process in this application:
[0115] S506: The first device displays a first control, where the first control is used to switch the camera called by the first device to the second camera of the second device.
[0116] In a possible implementation, the first control includes a camera flip control, and the camera flip control is used to implement flip switching between the first camera and the second camera. Figure 2A Control 210 in.
[0117] In another possible implementation, the first control includes an option control (or selection control) for the second camera. For example, the option control for the second camera is Figure 2B Control 220 in.
[0118] Optionally, when the first device runs the target application, it may first display the main interface of the target application, and after receiving an operation on the main interface of the target application, it may display the interface where the first control in the target application is located. Figure 6 , the target application is a remote conference application, and the first device displays Figure 6 The interface 60 shown includes a document display area, a function area, a video area 610 (a picture captured by the first camera), and a video area 620. The function area includes a camera management control 630, a microphone management control, and a document sharing control. After receiving a click operation on the video area 610, the first device may display an interface 20, which includes a camera flip control 210; or, after receiving an operation on the camera management control 630, the first device may display an interface 24, which includes an optional camera list, and the optional camera list includes a selection control 220 for the rear camera of the tablet 130.
[0119] S507: The first device obtains an operation on the first control.
[0120] S508: The first device establishes a second session with the second device, where the second session is carried on the first link and is used to keep the first link alive.
[0121] After the first device obtains the user's operation on the first control, the first device establishes a second session with the second device; when establishing the second session, the first device can still send a negotiation message corresponding to the second session to the second device, and accordingly, the second device receives the negotiation message and establishes a second session with the first device. The negotiation message includes a device identifier, a service type of the session, and service parameters of the session. Among them, the device identifier is used to indicate the peer device of the first device in the second session, the service type is used to indicate the type of data used to transmit in the second session (including video type, bytes type, etc.), and the service parameters include the keep-alive duration (or duration) of the second session, etc.
[0122] Optionally, the service type of the second session is different from the service type of the first session; the transmission resources required by the service type of the second session are smaller than the transmission resources required by the service type of the first session. For example, in the negotiation message corresponding to the second session, the service type of the second session can be set to bytes type, and the bandwidth required for bytes type session is much smaller than the bandwidth required for video type session. Based on this, the consumption of transmission resources can be reduced.
[0123] Optionally, the keep-alive duration of the second session is not set, or is set to the maximum value that a video-type session can survive, or the keep-alive duration of the second session is set to an empirical value, which is used to ensure that the first link can be kept alive (i.e., the first link is not disconnected) after the first session is closed and before a new video-type session is successfully created. The case where the keep-alive duration of the second session is set to an empirical value will be described in detail later.
[0124] S509: The first device sends a first closing request to the second device, where the first closing request is used to instruct the second device to close the first session and the first camera.
[0125] In the embodiment of the present application, after the first device obtains the user's operation on the first control, it can respond to the operation on the first control and simultaneously execute the above S502 and S503, that is, when the first device establishes the second session with the second device, it sends the first closing request to the second device. Correspondingly, the second device can receive the first closing request.
[0126] S510: The second device closes the first session and the first camera.
[0127] After receiving the first closing request, the second device closes the first session and the first camera. The process of closing the first session and the process of closing the first camera may be different, so the second device may close the first session and the first camera at the same time.
[0128] S511: The first device establishes a third session with the second device and sends a first open request, where the third session is carried on the first link, and the first open request is used to instruct the second device to start the second camera.
[0129] After the second device successfully closes the first session, the first device may establish a third session with the second device and send a first open request; the third session is used to transmit the second video stream captured by the second camera. Accordingly, the second device receives the first open request.
[0130] Wherein, when establishing the third session, the first device can still send a negotiation message corresponding to the third session to the second device, and accordingly, the third device receives the negotiation message and establishes a third session with the first device. The negotiation message includes a device identifier, a service type of the session, and service parameters of the session. Wherein, the device identifier is used to indicate the peer device of the first device in the third session, the service type is used to indicate the type of data used for transmission in the third session (including video type, bytes type, etc.), and the service parameters include the keep-alive duration (or duration) of the second session, etc. For example, the device identifier is the identifier of the third device, the service type is the video type, and the service parameters include that the keep-alive duration of the third session is not set, or is set to the maximum value that a video type session can survive, etc., to ensure reliable transmission of the second video stream.
[0131] Optionally, when executing the above S508 to set the keep-alive time of the second session to an empirical value, the empirical value is greater than or equal to the time interval between the moment when the second session is successfully established and the moment when the third session is successfully established. For example, the moment when the second session is successfully established is t1, and the moment when the third session is successfully established is t2. As can be seen from the above content, in the time period from t1 to t2, the first session on the first link is disconnected. Therefore, if the second session does not exist in the first link during the time period from the disconnection of the first session to the successful establishment of the third session, the first link will be disconnected, and the first device needs to re-establish the first link with the second device before establishing the third session with the second device. It can be seen that when the keep-alive time of the second session is greater than or equal to the time interval from t1 to t2, the second link will not be disconnected, which can effectively reduce the time for link reconstruction, thereby reducing the delay of camera switching.
[0132] S512: The second device turns on the second camera.
[0133] The second device can turn on the second camera after confirming that the first camera is successfully turned off. For details on how the second device can confirm that the first camera is successfully turned off, refer to the following Figure 7 The corresponding embodiments.
[0134] S513: The second device sends a second video stream captured by the second camera to the first device, and the second video stream is transmitted in the first session.
[0135] After the third session is successfully created and the second camera is successfully started, the second camera of the second device starts to capture the second video stream and sends the captured second video stream to the first device via the third session. Accordingly, the first device can receive the second video stream and display it according to the second video stream.
[0136] Optionally, the second device may display a prompt interface for camera use, for example, the second device displays Figure 2Aor Figure 2B Interface 23 in is used to prompt the user that the second device is using the second camera (such as the rear camera) to take pictures.
[0137] In this embodiment, for the scenario where the first device switches the called camera from the first camera of the second device to the second camera, a second session can be created between the first device and the second device to keep the first link alive before sending the first shutdown request to the second device. Based on this, even after the second device turns off the first camera and disconnects the first session with the first device, the first link will not be disconnected. Accordingly, there is no need to create a new link to carry the third session before the second device turns on the second camera and creates the third session. It can be seen that the present application can reduce the delay of camera switching by omitting the disconnection and reconstruction steps of the first link.
[0138] Previous Figure 5 The process of the camera switching method provided in the embodiment of the present application is described below. Figure 4A and Figure 4B The software module in further illustrates the camera switching method provided by the embodiment of the present application. Among them, the interaction between the application layer and the application framework layer is mainly introduced, and the calling process of the application layer and the application framework to the hardware abstraction layer and the kernel layer is not repeated.
[0139] See also Figure 7 , Figure 7 This is a flowchart of software interaction in a camera switching method provided in an embodiment of the present application. Among them, S701 to S709 are steps performed before the camera is switched when the camera is called across devices; the subsequent steps of S709 are steps performed when the camera is switched. The following first introduces S701 to S709:
[0140] S701: A third-party application of a first device obtains an operation of opening a first camera.
[0141] The implementation of S701 may refer to the corresponding description in S501.
[0142] S702: The third-party application of the first device sends a request to open the first camera to the camera management service of the first device.
[0143] Accordingly, the camera management service of the first device receives a request to open the first camera.
[0144] S703: The camera management service of the first device establishes a first session with the camera management service of the second device.
[0145] After the camera management service of the first device receives the opening request of the first camera, the camera management service of the first device may send a negotiation message to the camera management service of the second device to establish a first session. For example, the negotiation message may include the following content:
[0146] open P2P session1 Dv Service{
[0147] std:string DeviceID / / device identification
[0148] Dv ServiceType / / Service type of the session
[0149] std:<string,string> / / Service parameters for the session
[0150] }
[0151] For example, the device identifier is the identifier of the second device, the service type is the video type, and the service parameters include that the keep-alive duration of the first session is not set, or is set to the maximum value that can be allowed to survive for a video type session, etc., to ensure reliable transmission of the first video stream. The specific implementation of S703 can refer to the corresponding description of S502, which is not repeated here.
[0152] S704: The camera management service of the first device sends a request to open the first camera to the camera business service of the first device.
[0153] After the camera management service of the first device receives the opening request of the first camera, the camera management service of the first device forwards the opening request of the first camera to the camera business service of the first device. It can be understood that the camera management service of the first device can execute S703 and S704 at the same time. Accordingly, the camera business service of the first device receives the opening request of the first camera.
[0154] S705: The camera service of the first device sends a request to open the first camera to the camera service of the second device.
[0155] After the camera service of the first device receives the opening request of the first camera, it can send it to the camera service of the second device through the control link between the first device and the second device.
[0156] Optionally, the camera service of the first device records the service status of the first camera. When receiving the opening request of the first camera, the camera service of the first device may update the service status of the first camera from the no-service status to the service status.
[0157] S706: The camera business service of the second device sends an instruction to open the first camera to the camera management service of the second device.
[0158] After receiving the opening request of the first camera, the camera business service of the second device may send an opening instruction of the first camera to the camera management service of the second device. The opening instruction may be a Stream On instruction for instructing to start the first camera.
[0159] Optionally, the camera business service of the second device may first obtain user authorization, and then send an opening instruction of the first camera to the camera management service of the second device after obtaining the user authorization.
[0160] Optionally, the camera service of the second device records the service status of the first camera, and the camera service of the second device may update the service status of the first camera from an out-of-service state to a service state.
[0161] S707: The camera management service of the second device starts the first camera.
[0162] After receiving the opening instruction of the first camera, the camera management service of the second device can start the first camera.
[0163] S708. The camera management service of the second device sends the first video stream to the camera management service of the first device.
[0164] After the first camera is successfully started, the first camera collects the first video stream. The camera management service of the second device can send the first video stream to the camera management service of the first device through the first session. Correspondingly, the camera management service of the first device receives the first video stream.
[0165] S709: The camera management service of the first device sends a first video stream to a third-party application of the first device.
[0166] After receiving the first video stream, the camera management service of the first device may send it to a third-party application of the first device for display.
[0167] Here is an introduction to S709 and its subsequent steps:
[0168] S710. A third-party application of a first device displays a first control.
[0169] S711. A third-party application of the first device obtains an operation on the first control.
[0170] The implementation of S710 may refer to the corresponding description in S506 , and the implementation of S711 may refer to the corresponding description in S507 .
[0171] S712. The third-party application of the first device sends a first closing request to the camera management service of the first device.
[0172] The third-party application of the first device sends a first closing request to the camera management service of the first device in response to the operation on the first control; the first closing request is used to instruct to close the first camera and the first session.
[0173] Accordingly, the camera management service of the first device receives the first shutdown request. After receiving the first shutdown request, the camera management service of the first device may execute S713 and S714 simultaneously.
[0174] S713: The camera management service of the first device establishes a second session with the camera management service of the second device.
[0175] After the camera management service of the first device receives the first shutdown request, the camera management service of the first device may send a negotiation message corresponding to the second session to the camera management service of the second device to establish the second session. For example, in the negotiation message corresponding to the second session, the service type of the second session may be set to bytes type, and the bandwidth occupied by the bytes type session is much smaller than the bandwidth occupied by the video type session. And the keep alive time of the second session is an empirical value, which may be 5s, and is used to keep the first link alive for 5s after the second session is successfully established.
[0176] S714: The camera management service of the first device sends a first closing request to the camera business service of the first device.
[0177] Correspondingly, the camera business service of the first device receives the first closing request.
[0178] S715: The camera service of the first device sends a first closing request to the camera service of the second device.
[0179] After the camera service of the first device receives the first shutdown request, it can send it to the camera service of the second device through the control link between the first device and the second device. Accordingly, the camera service of the second device receives the first shutdown request.
[0180] Optionally, the camera service of the first device records the service status of the first camera. When receiving the first shutdown request, the camera service of the first device may update the service status of the first camera from a service status to a non-service status.
[0181] S716: The camera business service of the second device updates the service state of the first camera from a service state to a non-service state.
[0182] S717: The camera business service of the second device sends a first instruction to the camera management service of the second device.
[0183] The first instruction is a Tear down instruction, which is used to instruct to close the first camera and the first session.
[0184] S718: The camera management service of the second device detects whether the first instruction is received.
[0185] In the present application, after the camera management service of the second device establishes a second session with the camera management service of the first device, the camera management service of the second device will continue to determine whether it has received the first instruction sent by the camera service service of the second device. If the first instruction is received, the second device executes S719-1-1 and S719-1-2; if the first instruction is not received, the second device executes S719-2-1, S719-2-2, S719-2-3, and S719-2-4.
[0186] Optionally, the camera management service of the second device further detects whether an instruction indicating abnormal session termination is received. If an instruction indicating abnormal session termination is received, the first camera and the first session may be closed, thereby ensuring communication security.
[0187] S719-1-1. If the camera management service of the second device receives the first instruction, it turns off the first camera.
[0188] S719-1-2. If the camera management service of the second device receives the first instruction, it closes the first session.
[0189] Optionally, the process of closing the first camera is different from the process of closing the first session, so when the camera management service of the second device receives the first instruction, it can execute S719-1-1 and S719-1-2 at the same time.
[0190] S719-2-1. If the camera management service of the second device does not receive the first instruction, it queries the camera business service of the second device for the service status of the first camera.
[0191] S719-2-2. The camera business service of the second device sends the service status of the first camera to the camera management service of the second device.
[0192] The camera business service of the second device may send the service status of the first camera updated in S716 (ie, no service status) to the camera management service of the second device.
[0193] S719-2-3. If the service status of the first camera is out of service, the camera management service of the second device turns off the first camera.
[0194] S719-2-4: If the service status of the first camera is out of service, the camera management service of the second device and the camera management service of the first device close the first session.
[0195] Optionally, when the service status of the first camera is out of service, S719-2-3 and S719-2-4 may be executed simultaneously.
[0196] Optionally, if the service status of the first camera is in service status, it means that the camera service of the second device has not executed S716, which means that the camera service of the second device may not receive the first shutdown request due to transmission abnormality or other problems. In this case, the camera management service of the second device maintains the first session and keeps the first camera started. Based on this, the first camera and the first session can be kept consistent with the service status of the first camera.
[0197] S720. The third-party application of the first device sends a first opening request to the camera management service of the first device.
[0198] The first opening request is used to instruct to open the second camera. Accordingly, the camera management service of the first device receives the first opening request. After receiving the first closing request, the camera management service of the first device may simultaneously execute S721 and S722.
[0199] S721. The camera management service of the first device establishes a third session with the camera management service of the second device.
[0200] After the camera management service of the first device receives the first open request, the camera management service of the first device may send a negotiation message corresponding to the third session to the camera management service of the second device to establish the third session. For example, the device identifier is the identifier of the second device, the service type is the video type, and the service parameters include that the keep-alive time of the third session is not set, or is set to the maximum value that can be allowed to survive for a video type session, etc., to ensure reliable transmission of the second video stream.
[0201] S722: The camera management service of the first device sends a first opening request to the camera service service of the first device.
[0202] Accordingly, the camera business service of the first device receives the first opening request.
[0203] S723: The camera service of the first device sends a first opening request to the camera service of the second device.
[0204] After the camera service of the first device receives the first opening request, it can send it to the camera service of the second device through the control link between the first device and the second device. Accordingly, the camera service of the second device receives the first opening request.
[0205] Optionally, the camera service of the first device records the service status of the second camera. When receiving the first opening request, the camera service of the first device may update the service status of the second camera from the no service status to the service status.
[0206] S724: The camera business service of the second device sends a second instruction to the camera management service of the second device.
[0207] After the camera service of the second device receives the first opening request, a second instruction may be sent to the camera management service of the second device. The second instruction is a Stream On instruction, which is used to instruct to open the second camera. Accordingly, the camera management service of the second device receives the second instruction.
[0208] Optionally, the camera business service of the second device also updates the service state of the second camera from an out-of-service state to a service state.
[0209] S725. The camera management service of the second device queries the working status of the first camera and the working status of the second camera.
[0210] The camera management service of the second device records the working status of the first camera and the working status of the second camera. The working status is an open state or a closed state. Since the camera management service of the second device needs a certain amount of time to execute the closing of the first camera, after the camera management service of the second device receives the second instruction, it can first query the working status of the first camera and the working status of the second camera, and then execute S726-1, and the latter executes S726-2.
[0211] S726-1. If the working state of the first camera is the start state and the working state of the second camera is the stop state, the camera management service of the second device turns off the first camera and then turns on the second camera.
[0212] S726-2: If the working state of the first camera is off and the working state of the second camera is off, the camera management service of the second device starts the second camera.
[0213] S727. The camera management service of the second device sends a second video stream to the camera management service of the first device.
[0214] After the second camera is successfully started and the third session is successfully created, the second camera captures the second video stream. The camera management service of the second device can send the second video stream to the camera management service of the first device through the third session. Correspondingly, the camera management service of the first device receives the second video stream.
[0215] S728. The camera management service of the first device sends the second video stream to the third-party application of the first device.
[0216] After receiving the second video stream, the camera management service of the first device may send it to a third-party application of the first device for display, thereby achieving the effect of switching from the first camera to the second camera.
[0217] It should be noted that if the keep-alive time of the second session is set to an empirical value, and the empirical value is greater than or equal to the time interval between the time when the second session is successfully established and the time when the third session is successfully established, the second session is automatically closed when the keep-alive time is reached. For example, if the empirical value is equal to the time interval between the time when the second session is successfully established and the time when the third session is successfully established, and no abnormality occurs during the establishment of the third session, then ideally, the second session is automatically closed when the third session is successfully established.
[0218] In this embodiment, for the scenario where the first device switches the called camera from the first camera of the second device to the second camera, a second session can be created between the first device and the second device to keep the first link alive before sending the first shutdown request to the second device. Based on this, the delay of camera switching can be reduced by omitting the disconnection and reconstruction steps of the first link. At the same time, when the second device turns off the first camera, it is determined whether the first camera needs to be turned off through the first instruction and the service status of the first camera, which is conducive to ensuring that the working status of the first camera is consistent with the service status of the first camera. In addition, when the second device turns on the second camera, by querying the working status of the two cameras, it can be ensured that no errors or abnormalities will occur when the second camera is turned on, thereby improving the accuracy of camera switching.
[0219] The embodiment of the present application also provides an electronic device, which may include: one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes each function or step performed by the first device / second device in the above method embodiment.
[0220] An embodiment of the present application also provides a camera switching device, which includes a function / unit for executing the first device / second device in the above embodiment.
[0221] The present application also provides a chip system, such as Figure 8 As shown, the chip system 800 includes at least one processor 801 and at least one interface circuit 802. The processor 801 and the interface circuit 802 can be interconnected via lines. For example, the interface circuit 802 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 802 can be used to send signals to other devices (such as processor 801). Exemplarily, the interface circuit 802 can read instructions stored in the memory and send the instructions to the processor 801. When the instructions are executed by the processor 801, the electronic device can execute the various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.
[0222] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes each function or step executed by the mobile phone in the above method embodiment.
[0223] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the mobile phone in the above method embodiment.
[0224] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes each function or step performed by the mobile phone in the above method embodiment.
[0225] Among them, the electronic device, communication system, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.
[0226] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0227] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0228] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.
[0229] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0230] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.
[0231] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.
Claims
1. A camera switching method, It is characterized in that Applied to a first device, the method includes: In the case where the first device receives a first video stream captured by a first camera of a second device, a first control is displayed; the first device establishes a first link with the second device; the first video stream is transmitted via a first session carried in the first link; the first control is used to switch the camera called by the first device to the second camera of the second device; In response to a user operation on the first control, establishing a second session with the second device; the second session is carried on the first link and is used to keep the first link alive; Sending a first closing request to the second device; the first closing request is used to instruct the second device to close the first session and close the first camera; After the first session is successfully closed, a third session is established with the second device, and a first opening request is sent to the second device; the first opening request is used to instruct the second device to start the second camera; the third session is carried on the first link; A second video stream is received; the second video stream is captured by the second camera of the second device and transmitted to the first device via the third session.
2. The method according to claim 1, It is characterized in that The keep-alive duration of the second session is greater than or equal to a time interval between a time when the second session is successfully established and a time when the third session is successfully established.
3. The method according to claim 1 or 2, It is characterized in that The second device is a device within the near field communication range of the first device and / or a device logged in with the same user account as the first device.
4. The method according to claim 1 or 2, It is characterized in that The first control includes a camera flip control, and the camera flip control is used to implement flip switching between the first camera and the second camera; or, the first control includes an option control of the second camera.
5. A camera switching method, It is characterized in that Applied to the second device, the method includes: In the case of sending a first video stream captured by a first camera of the second device to the first device, establishing a second session with the first device; the first device and the second device establish a first link; the first video stream is transmitted via the first session carried in the first link; the second session is carried on the first link and is used to keep the first link alive; receiving a first closing request sent by the first device, and closing the first session and the first camera in response to the first closing request; After the first session is successfully closed, establishing a third session with the second device; receiving a first opening request sent by the first device, and starting a second camera in response to the first opening request; A second video stream is sent to the first device; the second video stream is captured by the second camera and transmitted to the first device via the third session.
6. The method according to claim 5, It is characterized in that The keep-alive duration of the second session is greater than or equal to a time interval between a time when the second session is successfully established and a time when the third session is successfully established.
7. The method according to claim 5 or 6, It is characterized in that The first device includes a camera management service, and the second device includes a camera business service and a camera management service; The establishing a second session with the first device includes: The camera management service of the second device establishes a second session with the camera management service of the first device; The closing of the first session and the first camera includes: After the camera management service of the second device establishes a second session with the camera management service of the first device, the camera management service of the second device detects whether a first instruction is received; the first instruction is used to instruct to close the first session and the first camera; If the camera management service of the second device receives the first instruction, the camera management service of the second device closes the first session and the first camera; Alternatively, if the camera management service of the second device does not receive the first instruction, the camera management service of the second device queries the camera business service about the service status of the first camera; if the service status of the first camera is out of service, the camera management service of the second device closes the first session and the first camera.
8. The method according to claim 7, It is characterized in that Before closing the first session and the first camera, the method further includes: In response to the first shutdown request sent by the first device, the camera business service of the second device updates the service state of the first camera from the service state to the no service state; The camera business service of the second device sends the first instruction to the camera management service of the second device.
9. The method according to claim 7, It is characterized in that The step of starting the second camera in response to the first opening request sent by the first device includes: In response to the first opening request sent by the first device, the camera business service of the second device sends a second instruction to the camera management service of the second device; the second instruction is used to instruct to start the second camera; The camera management service of the second device queries the working status of the first camera and the working status of the second camera in response to the second instruction; If the working state of the first camera is the start state and the working state of the second camera is the stop state, turn off the first camera, and after the first camera is successfully turned off, start the second camera; Alternatively, if the working state of the first camera is off and the working state of the second camera is off, the second camera is started.
10. The method according to claim 5, It is characterized in that The second device is a device within the near field communication range of the first device and / or a device logged in with the same user account as the first device.
11. An electronic device, It is characterized in that include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-4 or claims 5-10.
12. A communication system, It is characterized in that The communication system comprises the first device and the second device as claimed in claims 1-10.
13. A chip system, applied to electronic equipment, It is characterized in that The chip system includes at least one processor and an interface, wherein the interface is used to receive computer instructions and transmit them to the at least one processor; the at least one processor executes the computer instructions so that the electronic device executes the method described in any one of claims 1-4 or claims 5-10.
14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 4 or claims 5 to 10.
