Video conference switching method and video conference system

Abstract

The invention provides a video conference switching method and a video conference system. The video conference switching method is suitable for a video conference system comprising a first camera device and a second camera device, and comprises the following steps. A first captured image of a first camera device and a second captured image of a second camera device are acquired. And performing image object identification on the first shot image and the second shot image by using a neural network model. And switching the video source of the video conference to the first camera device or the second camera device according to the model identification result of the first shot image and the model identification result of the second shot image. And performing a video conference according to the image of the video source. Therefore, the convenience of the video conference can be improved.

Description

Technical Field

[0001] The present invention relates to a video conference method, and in particular to a video conference switching method and a video conference system. Background Art

[0002] Due to the advancement of science and technology, the trend of globalization, the need for remote work, distance learning and other factors, it has become quite common for many people to use video conferencing software to conduct remote work or online courses at home. Therefore, in order to allow users to conduct video conferencing more conveniently and meet the needs of various operating scenarios, how to improve video conferencing technology is a topic of great concern to technicians in this field. For example, in some application scenarios, users can use the content captured by more than one camera device to generate video conferencing images, but users need to manually switch camera devices during the video conferencing process to obtain video conferencing images that meet current needs. This operation process is quite cumbersome and inconvenient. Summary of the invention

[0003] The present invention provides a video conference switching method and a video conference system, which can automatically switch camera devices during a video conference, thereby improving the convenience of the video conference.

[0004] The embodiment of the present invention provides a video conference switching method, which is applicable to a video conference system including a first camera device and a second camera device, and includes the following steps. A first captured image of the first camera device and a second captured image of the second camera device are obtained. Image object recognition is performed on the first captured image and the second captured image respectively using a neural network model. Based on the model recognition result of the first captured image and the model recognition result of the second captured image, the video source of the video conference is switched to the first camera device or the second camera device. A video conference is performed based on the image of the video source.

[0005] An embodiment of the present invention provides a video conferencing system, which includes a first camera device, a second camera device, a storage device, and a processor. The storage device stores a plurality of instructions. The processor is coupled to the first camera device, the second camera device, and the storage device, and executes the plurality of instructions to: obtain a first captured image of the first camera device and a second captured image of the second camera device; perform image object recognition on the first captured image and the second captured image respectively using a neural network model; switch the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image; and conduct a video conference according to the image of the video source.

[0006] Based on the above, in an embodiment of the present invention, during the video conference, a neural network model can be used to perform image object recognition on the images captured by multiple camera devices, so that one of the multiple camera devices can be automatically switched to the video source of the video conference according to the model recognition results of the captured images. Therefore, the tedious steps of manually switching camera devices during the video conference can be saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of a video conferencing system according to an embodiment of the present invention;

[0008] Figure 2 is a flow chart of a method for switching a video conference according to an embodiment of the present invention;

[0009] Figure 3 is a schematic diagram of a video conference switching method according to an embodiment of the present invention;

[0010] Figure 4 is a flow chart of a method for switching a video conference according to an embodiment of the present invention;

[0011] Figure 5 is a flow chart of a method for switching a video conference according to an embodiment of the present invention;

[0012] Figure 6 is a flow chart of a method for switching a video conference according to an embodiment of the present invention;

[0013] Figure 7 is a flow chart of a method for switching a video conference according to an embodiment of the present invention;

[0014] Fig. 8A and Figure 8B is a schematic diagram of a video conference scenario according to an embodiment of the present invention.

[0015] Description of Reference Numerals

[0016] 10: Video conferencing system;

[0017] 110: a first camera device;

[0018] 120: second camera device;

[0019] 160: display;

[0020] 130: storage device;

[0021] 140: processor;

[0022] 150,122: microphone device;

[0023] 121: motion sensor;

[0024] 130a:DMFT;

[0025] 130b: application layer program;

[0026] 130c: Video conferencing application;

[0027] F1, F2: shooting picture;

[0028] U1: user;

[0029] NB1: Laptop;

[0030] M1: preset neural network model;

[0031] S210~S240, S410~S460, S510~S560, S610~S660, S710~S780: steps. DETAILED DESCRIPTION

[0032] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0033] Figure 1 is a schematic diagram of a video conferencing system according to an embodiment of the present invention. Figure 1 The video conferencing system 10 may include a first camera device 110, a second camera device 120, a storage device 130, a processor 140, a microphone device 150, and a display 160. The video conferencing system 10 may conduct a video conference with other user terminal devices through a network. The above network may include any of a variety of wireless and / or wired networks. For example, the network may include any combination of public and / or private networks, local area networks, and / or wide area networks, etc.

[0034] In some embodiments, the video conferencing system 10 may be implemented as a computer device including a built-in first camera device 110 and a second camera device 120 externally connected to the computer device. Specifically, the computer device may include the first camera device 110, a storage device 130, a processor 140, a microphone device 150, and a display 160. The computer device may be, for example, a tablet computer, a laptop computer, or a desktop computer, but the present invention is not limited thereto. In other words, the first camera device 110 may be a built-in camera built into the computer device. In addition, the second camera device 120 may be connected to the computer device by wire or wirelessly.

[0035] Alternatively, in some embodiments, the video conferencing system 10 may also be implemented by integrating the first camera device 110, the second camera device 120, the storage device 130, the processor 140, the microphone device 150, and the display 160 into a single electronic device. For example, the video conferencing system 10 may be implemented as a computer device with multiple built-in camera modules, but the present invention is not limited thereto. The computer device may be, for example, a tablet computer, a laptop computer, or a desktop computer, but the present invention is not limited thereto.

[0036] Alternatively, in some embodiments, the video conferencing system 10 may be implemented as including a computer device and a first camera device 110 and a second camera device 120 externally connected to the computer device. Specifically, the computer device may include a storage device 130, a processor 140, a microphone device 150, and a display 160, which may be a tablet computer, a laptop computer, or a desktop computer, but the present invention is not limited thereto. In addition, the first camera device 110 and the second camera device 120 may be connected to the computer device by wire or wirelessly.

[0037] In some embodiments, the first camera device 110 and the second camera device 120 may be connected to the processor 140 via a wireless communication interface. The wireless transmission interface is, for example, a Bluetooth interface, a WiFi interface, or other wireless transmission interfaces. Alternatively, the first camera device 110 and the second camera device 120 may also be connected to the processor 150 via a wired transmission interface. The wired transmission interface is, for example, a USB interface or other wired transmission interfaces.

[0038] In some embodiments, the second camera device 120 may be a mobile electronic device with a photography function, such as a mobile phone, a tablet computer, or a digital camera, etc., and the present invention is not limited thereto. In some embodiments, the second camera device 120 may be an external camera or a webcam, etc., and the present invention is not limited thereto. For example, the video conferencing system 10 may be composed of a laptop or a desktop computer and an external camera. Alternatively, the video conferencing system 10 may be composed of a laptop or a desktop computer and a smart phone.

[0039] In some embodiments, the second camera device 120 may include a motion sensor 121 and a microphone device 122. The motion sensor 121 is, for example, a gyroscope, an acceleration sensor, or a gravity sensor, etc. The motion sensor 121 may be used to sense the movement track, motion, or posture of the second camera device 120.

[0040] The microphone device 122 and the microphone device 150 are audio generating devices that can convert sound waves into electronic sound signals. For example, the microphone device 122 and the microphone device 150 can be used to receive sound signals generated by a user speaking.

[0041] The first camera device 110 and the second camera device 120 can be used to capture continuous images to generate a video sequence, and include a camera lens having a lens and a photosensitive component. The photosensitive component is used to sense the intensity of light entering the lens to generate an image. The photosensitive component can be, for example, a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) component or other components, and the present invention is not limited thereto.

[0042] The storage device 130 is used to store data such as files, images, instructions, program codes, software components, etc. It can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices, integrated circuits or combinations thereof.

[0043] The display 160 is, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or other types of displays, which are not limited in the present invention. The display 160 can be used to display the video conference screen of the video conference.

[0044] The processor 140 is, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), an image signal processor (ISP), a graphics processing unit (GPU), or other similar devices, integrated circuits, or combinations thereof.

[0045] The processor 140 can execute the program code, software / firmware module, instruction, etc. recorded in the storage device 130 to implement the video conference switching method of the embodiment of the present disclosure. In other words, the processor 140 can be configured to perform the corresponding operations described below.

[0046] More specifically, in an embodiment of the present invention, the storage device 130 may record a device media foundation transform (DMFT) 130a, an application layer program 130b, and a video conferencing application 130c, and the processor 140 may execute the DMFT 130a, the application layer program 130b, and the video conferencing application 130c. In addition, the storage device 130 may record a plurality of trained preset neural network models M1. The model parameters and model architectures of these trained preset neural network models M1 are recorded in the storage device 130.

[0047] The DMFT 130a is executed in the user mode of the operating system. The DMFT 130a is a user mode extension. Specifically, the DMFT 130a is a special media foundation transform (MFT). The DMFT 130a can provide some frame processing functions for the video conferencing application 130c, such as video editing and image stabilization. More specifically, the DMFT 130a can be encapsulated in the camera driver stack and used to perform post-image processing on the video streams obtained from the first camera device 110 and the second camera device 120. The DMFT 130a can have one or more input data streams and one or more output data streams.

[0048] In the embodiment of the present invention, the application layer program 130b may be an external application program independent of the operation process of the DMFT 130a, which may be used to obtain the captured images transmitted in the processing pipeline of the DMFT 130a. The application layer program 130b may use these preset neural networks M1 to analyze the captured image contents of multiple camera devices (i.e., the first camera device 110 and the second camera device 120), so as to control the DMFT 130a to output the video sequence captured by a certain camera device to the video conferencing application 130c according to the captured image contents of the multiple camera devices. Based on this, the video source of the video conference may be automatically switched between the multiple camera devices.

[0049] Figure 2 is a flow chart of a switching method according to an embodiment of the present invention. Figure 2 The method of this embodiment is applicable to the video conferencing system 10 of the above embodiment. The following describes the detailed steps of this embodiment in conjunction with the various components in the video conferencing system 10.

[0050] In step S210, during the video conference, the processor 140 may obtain the first captured image of the first camera device 110 and the second captured image of the second camera device 120. Specifically, the processor 140 may execute the video conference application 130c to conduct the video conference. During the video conference, the processor 140 may transmit the video sequence captured by the first camera device 110 or the second camera device 120 to other terminal devices via the network, so that the conference participants of the video conference can see the video screen of the user of the video conference system 10. More specifically, during the video conference, the DMFT 130a may transmit the video sequence captured by the first camera device 110 or the second camera device 120 to the video conference application 130c.

[0051] In some embodiments, the processor 140 may utilize the application layer program 130b to obtain the first captured image of the first camera device 110 and the second captured image of the second camera device 120 through the DMFT 130a. In some embodiments, the DMFT 130a may provide an application programming interface (API) to allow the application layer program 130b to obtain the first captured image and the second captured image from the shared memory.

[0052] In step S220, the processor 140 uses a neural network model to perform image object recognition on the first captured image and the second captured image respectively. The neural network model may be a trained convolutional neural network model. From another point of view, the neural network model is an object recognition model, which is used to identify one or more specific objects from an input image. The neural network model is, for example, R-CNN, Fast R-CNN, Faster R-CNN, YOLO or SSD, etc., which are used for object detection in a convolutional neural network (CNN) model. The present invention does not limit the network architecture used by the neural network model. After the neural network model performs image object recognition on the first captured image and the second captured image respectively, the model recognition result of the first captured image and the model recognition result of the second captured image may respectively include recognition results of specific objects. The above-mentioned specific objects are, for example, human faces, animals, product introductions or other objects, etc., and the present invention is not limited to this. In some embodiments, when the processor 140 detects a specific object from a captured image (i.e., the first captured image or the second captured image) using a neural network model, the model recognition result of the captured image (i.e., the first captured image or the second captured image) may include the recognition confidence level of the specific object, the object category, the object location, etc.

[0053] In some embodiments, the neural network model can be trained based on multiple training images labeled with specific objects. More specifically, after planning the convolutional neural network architecture of the neural network model, the training images and the classification answer information of each training image must be used to determine the weight information in the neural network model, so as to train a neural network model that can recognize specific objects from the input image.

[0054] In step S230, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 according to the model recognition result of the first captured image and the model recognition result of the second captured image. That is, according to the image content of the first captured image and the image content of the second captured image, the processor 140 switches the video source of the video conference from the first camera device 110 to the second camera device 120. Alternatively, according to the image content of the first captured image and the image content of the second captured image, the processor 140 switches the video source of the video conference from the second camera device 120 to the first camera device 110. In some embodiments, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 by controlling the output data stream of the DMFT 130a. When the processor 140 switches the video source of the video conference to the first camera device 110, the DMFT 130a transmits the video sequence generated by the first camera device 110 to the video conference application 130c. Alternatively, when the processor 140 switches the video source of the video conference to the second camera device 120 , the DMFT 130 a transmits the video sequence generated by the second camera device 120 to the video conference application 130 c .

[0055] In some embodiments, the processor 140 can perform image object recognition on the first captured image and the second captured image simultaneously, thereby saving computing time and achieving a real-time switching effect. For example, the first computing core of the processor 140 is responsible for performing image object recognition on the first captured image, while the second computing core of the processor 140 is responsible for performing image object recognition on the second captured image.

[0056] In step S240, the processor 140 conducts a video conference based on the image of the video source. When conducting a video conference, the processor 140 may transmit the image of the video source to other conference participants of the video conference via the network. That is, when the video source of the video conference is switched to the first camera device 110, the video sequence captured by the first camera device 110 will be transmitted to other conference participants of the video conference. Alternatively, when the video source of the video conference is switched to the second camera device 110, the video sequence captured by the second camera device 110 will be transmitted to other conference participants of the video conference.

[0057] Figure 3is a schematic diagram of a video conference switching method according to an embodiment of the present invention. Figure 3 , DMFT 130a obtains the video sequences captured by the first camera device 110 and the second camera device 120 respectively. The above video sequence includes multiple captured images. The application layer program 130b can obtain the first captured image of the first camera device 110 and the second captured image of the second camera device 120 from the DMFT 130a. The application layer program 130b can use a neural network model to perform image object recognition on the first captured image and the second captured image. According to the model recognition result of the first captured image and the model recognition result of the second captured image, the application layer program 130b can send a switching request instruction to the DMFT 130a. In response to the switching request instruction of the application layer program 130b, the DMFT 130a can choose to output the video sequence of the first camera device 110 or the video sequence of the second camera device 120 to the video conferencing software 130c, so that the video conferencing software 130c transmits the video sequence of the first camera device 110 or the video sequence of the second camera device 120 to other conference participants.

[0058] It can be seen that, according to the image content captured by different camera devices, the processor 140 can automatically switch the video source of the video conference to a specific camera device among the multiple camera devices. Therefore, the video screen of the user in the video conference can meet the conference situation requirements. In addition, based on different conference situation requirements, the processor 140 can automatically switch the video source of the video conference to a specific camera device among the multiple camera devices according to appropriate judgment conditions. The subsequent embodiments will be described with respect to different switching conditions.

[0059] Figure 4 is a flow chart of a method for switching a video conference according to an embodiment of the present invention. Figure 4 The method of this embodiment is applicable to the video conferencing system 10 of the above embodiment. The following describes the detailed steps of this embodiment in conjunction with the various components in the video conferencing system 10.

[0060] In step S410, the processor 140 starts a video conference. In step S420, the processor 140 obtains a first captured image of the first camera device 110 and a second captured image of the second camera device 120. The implementation and details of the above steps S410 to S420 are described in the above Figures 1 to 3 The embodiments of the present invention have been described in detail, so I will not go into details here.

[0061] In step S430, according to an application scenario setting, the processor 140 selects a neural network model from a plurality of preset neural network models M1. The plurality of preset neural network models M1 include an object recognition model or a face recognition model. The application scenario setting may be set by a user instruction issued by a user. For example, assuming that the application scenario is set to a face tracking mode, the processor 140 may select a face recognition model from a plurality of preset neural network models M1 to perform subsequent operations, and the face recognition model is used to perform face recognition on the first captured image and the second captured image. Assuming that the application scenario is set to a product introduction mode, the processor 140 may select an object recognition model from a plurality of preset neural network models M1 to perform subsequent operations, and the object recognition model is used to detect specific product objects on the first captured image and the second captured image.

[0062] In step S440, the processor 140 uses the neural network model to perform image object recognition on the first captured image and the second captured image respectively. In step S450, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 according to the model recognition result of the first captured image and the model recognition result of the second captured image. Here, step S450 can be implemented as steps S451 to S453.

[0063] In this embodiment, in step S451, the processor 140 determines whether the second captured image includes a specific object. For example, the processor 140 may input the second captured image into a neural network model to generate an object recognition result of the specific object. The object recognition result may include a recognition confidence of the specific object. When the recognition confidence of the specific object is greater than a critical value, the processor 140 may determine that the second captured image includes the specific object. When the recognition confidence of the specific object is not greater than a critical value, the processor 140 may determine that the second captured image does not include the specific object.

[0064] In step S452, in response to the neural network model detecting that the second captured image includes a specific object (step S451 is determined to be yes), the processor 140 switches the video source of the video conference to the second camera device 120. In step S453, in response to the neural network model not detecting that the second captured image includes a specific object (step S451 is determined to be no), the processor 140 switches the video source of the video conference to the first camera device 110. In step S460, the processor 140 conducts a video conference based on the image of the video source.

[0065] For example, the first camera device 110 may be a main camera built into a laptop computer, and the second camera device 120 may be a mobile phone device. When it is determined that the image captured by the mobile phone device includes a human face object, the processor 140 may switch the video source of the video conference to the mobile phone device. Conversely, when it is determined that the image captured by the mobile phone device does not include a human face object, the processor 140 may switch the video source of the video conference to the main camera built into the laptop computer.

[0066] Figure 5 is a flow chart of a method for switching a video conference according to an embodiment of the present invention. Figure 5 The method of this embodiment is applicable to the video conferencing system 10 of the above embodiment. The following describes the detailed steps of this embodiment in conjunction with the various components in the video conferencing system 10.

[0067] In step S510, the processor 140 starts the video conference. In step S520, the processor 140 obtains the first captured image of the first camera device 110 and the second captured image of the second camera device 120. In step S530, according to an application scenario setting, the processor 140 selects a neural network model from a plurality of preset neural network models M1. The plurality of preset neural network models M1 include object recognition models or face recognition models. The implementation and details of the above steps S510 to S540 are described in the above Figures 1 to 4 The embodiments of the present invention have been described in detail, so I will not go into details here.

[0068] In step S540, the processor 140 uses a neural network model to perform image object recognition on the first captured image and the second captured image respectively. In this embodiment, the processor 140 selects a face recognition model from a plurality of preset neural network models M1. In some embodiments, the face recognition model can recognize a face object and a plurality of face feature points (such as eye feature points, nose feature points, mouth feature points, etc.) from an input image. The processor 140 can calculate the face angle based on these face feature points. In some embodiments, the face recognition model can directly recognize a face object and a corresponding face angle from an input image. The face recognition model can be implemented using any face recognition technology known to those skilled in the art, and the present invention is not limited thereto.

[0069] In step S550, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 according to the model recognition result of the first captured image and the model recognition result of the second captured image. Here, step S550 can be implemented as steps S551 to S553. In this embodiment, the model recognition result includes a face object and a corresponding face angle.

[0070] In step S551, the processor 140 determines whether the facial angle of the facial object in the second captured image meets the angle condition. In some embodiments, the processor 140 determines whether a second angle difference between the facial angle of the facial object in the second captured image and a preset angle is smaller than a first angle difference between the facial angle of the facial object in the first captured image and the preset angle. That is, the processor 140 determines whether the facial angle of the facial object in the second captured image is closer to the preset angle. Alternatively, in some embodiments, the processor 140 determines whether the facial angle of the facial object in the second captured image falls within a specific angle range.

[0071] In step S552, in response to the neural network model detecting that the face angle of the face object in the second captured image meets the angle condition (step S551 is determined to be yes), the processor 140 switches the video source of the video conference to the second camera device 120. In step S553, in response to the neural network model detecting that the face angle of the face object in the second captured image does not meet the angle condition (step S551 is determined to be no), the processor 140 switches the video source of the video conference to the first camera device 110. In step S560, the processor 140 conducts a video conference based on the image of the video source.

[0072] For example, the first camera device 110 may be a main camera built into a laptop computer, and the second camera device 120 may be a mobile phone device. When the user's face is facing the mobile phone device, the processor 140 determines that the face angle of the face object in the second captured image meets the angle condition, and switches the video source of the video conference to the mobile phone device. On the contrary, when the user's face is not facing the mobile phone device, the processor 140 determines that the face angle of the face object in the second captured image does not meet the angle condition, and the processor 140 may switch the video source of the video conference to the main camera built into the laptop computer.

[0073] Figure 6 is a flow chart of a method for switching a video conference according to an embodiment of the present invention. Figure 6 The method of this embodiment is applicable to the video conferencing system 10 of the above embodiment. The following describes the detailed steps of this embodiment in conjunction with the various components in the video conferencing system 10.

[0074] In step S610, the processor 140 starts the video conference. In step S620, the processor 140 obtains the first captured image of the first camera device 110 and the second captured image of the second camera device 120. In step S630, according to an application scenario setting, the processor 140 selects a neural network model from a plurality of preset neural network models. The plurality of preset neural network models include object recognition models or face recognition models. The implementation and details of the above steps S610 to S630 are described in the above Figures 1 to 5The embodiments of the present invention have been described in detail, so I will not go into details here.

[0075] In step S640, the processor 140 uses a neural network model to perform image object recognition on the first captured image and the second captured image respectively. In this embodiment, the processor 140 selects a face recognition model from a plurality of preset neural network models M1. In some embodiments, the face recognition model can recognize a face object from an input image. In step S650, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 based on the model recognition result of the first captured image and the model recognition result of the second captured image. Here, step S650 can be implemented as steps S651 to S564.

[0076] It should be noted that in step S651, the processor 140 determines whether the motion state of the second camera device 120 conforms to a specific action based on the sensing data of the motion sensor 112. By using the motion sensor 121 of the second camera device 120, the processor 140 can detect the motion state and device posture of the second camera device 120 in the three-axis motion space, and determine whether the motion state of the second camera device 120 conforms to a specific action. For example, assuming that the second camera device 120 is a mobile phone device, the processor 140 can determine whether the second camera device 120 is lifted by the user to perform a shooting action based on the sensing data of the motion sensor 121.

[0077] In step S652, in response to the motion state of the second camera device 120 being consistent with the specific motion (step S651 is determined to be yes), the processor 140 determines whether the second captured image includes a human face object. In step S653, in response to the sensing data of the motion sensor 121 being consistent with the specific motion and the neural network model detecting that the second captured image includes a human face object (step S653 is determined to be yes), the processor 140 switches the video source of the video conference to the second camera device 120.

[0078] In step S654, in response to the sensing data of the motion sensor 121 not matching the specific motion or the neural network model not detecting that the second captured image includes a face object (step S651 is determined as No or step S652 is determined as No), the processor 140 switches the video source of the video conference to the first camera device 110. In step S660, the processor 140 conducts a video conference based on the image of the video source.

[0079] For example, the first camera device 110 may be a main camera built into a laptop computer, and the second camera device 120 may be a mobile phone device. When the user needs to leave his seat, the user may raise the mobile phone device to take a picture of his face. Then, the processor 140 may switch the video source of the video conference to the mobile phone device according to the sensing data of the motion sensor of the mobile phone device and the second captured image. Therefore, even if the user needs to leave his seat, other conference participants can still continue to see the user's facial image on the video conference screen.

[0080] Figure 7 is a flow chart of a method for switching a video conference according to an embodiment of the present invention. Figure 7 The method of this embodiment is applicable to the video conferencing system 10 of the above embodiment. The following describes the detailed steps of this embodiment in conjunction with the various components in the video conferencing system 10.

[0081] In step S710, the processor 140 starts a video conference. In step S720, the processor 140 obtains a first captured image of the first camera device 110 and a second captured image of the second camera device 120. In step S730, the processor 140 uses a neural network model to perform image object recognition on the first captured image and the second captured image respectively. In step S740, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 according to the model recognition result of the first captured image and the model recognition result of the second captured image. The implementation methods and details of the above steps S710 to 740 are described in the above Figures 1 to 6 The embodiments of the present invention have been described in detail, so I will not go into details here.

[0082] In step S750 , the processor 140 receives a sound signal through the microphone device 122 or the microphone device 150 . In some embodiments, the processor 140 receives a first sound signal through the microphone device 150 and receives a second sound signal through the microphone device 122 .

[0083] In step S760, the processor 140 detects the sound characteristics of the sound signal through another neural network model. The sound characteristics include voiceprints, etc. In some embodiments, the processor 140 detects the sound characteristics of the second sound signal of the microphone device 122 through another neural network model, and detects the sound characteristics of the first sound signal of the microphone device 150 through another neural network model.

[0084] In step S770, the processor 140 switches the video source of the video conference to the first camera device 110 or the second camera device 120 according to the sound characteristics. In some embodiments, the processor 140 may determine whether the voiceprint of the second sound signal of the microphone device 122 matches the user's voiceprint. When the voiceprint of the second sound signal matches the user's voiceprint, the processor 140 may determine whether the sound collection distance of the second sound signal is less than a preset distance. When the sound collection distance of the second sound signal is less than the preset distance, the processor 140 may switch the video source of the video conference to the second camera device 120. On the contrary, when the voiceprint of the second sound signal of the microphone device 122 does not match the user's registered voiceprint or the sound collection distance of the second sound signal is greater than the preset distance, the processor 140 may switch the video source of the video conference to the first camera device 150. In step S750, the processor 140 conducts a video conference based on the image of the video source.

[0085] For example, the first camera device 110 may be a main camera built into a laptop computer, and the second camera device 120 may be a mobile phone device. When the user must leave the seat, the user can carry the mobile phone device to continue taking photos and receiving audio. Therefore, the processor 140 may switch the video source of the video conference to the mobile phone device according to the sound characteristics of the sound signal received by the microphone device of the mobile phone device. Therefore, even if the user must leave the seat, other conference participants can still continue to see the user's facial image on the video conference screen.

[0086] Fig. 8A and Figure 8B FIG. 1 is a schematic diagram of a video conference scenario according to an embodiment of the present invention. Fig. 8A and Figure 8B , assuming that the first camera device 110 is a built-in main camera of the notebook computer NB1, and the second camera device 120 is a mobile phone device mounted at a remote location of the seat. Fig. 8A , user U1 uses laptop computer NB1 to conduct video conference at his seat, and the first camera device 110 of laptop computer NB1 is first set as the video source of the video conference. Then, the picture F1 captured by the first camera device 110 is transmitted to other conference participants of the video conference. Figure 8B , the user U1 leaves his seat and moves to the second camera device 120, and displays the notebook computer NB1 as an introduction product. Therefore, in response to the second captured image including a face and the corresponding face angle meeting the angle condition, the video source of the video conference can be switched from the first camera device 110 to the second camera device 120. Therefore, the captured image F2 captured by the second camera device 120 will be transmitted to other conference participants of the video conference.

[0087] In summary, in the embodiments of the present invention, when a multi-camera device is used for video conferencing, the user's video screen can be automatically switched according to the conference context requirements, thereby effectively enhancing the communication efficiency and fluency of the video conference. In addition, according to different application context requirements, an appropriate neural network model can be selected to recognize the image content, thereby flexibly switching the video source of the video conference according to the actual context requirements.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video conference switching method, characterized in that: A video conferencing system including a first camera device and a second camera device, comprising: Acquire a first image captured by the first camera device and a second image captured by the second camera device; Using a neural network model to perform image object recognition on the first captured image and the second captured image respectively; Switching a video source of the video conference to the first camera device or the second camera device according to a model recognition result of the first captured image and a model recognition result of the second captured image; and The video conference is conducted based on the image of the video source.

2. The video conference switching method according to claim 1, characterized in that: Before the step of using the neural network model to perform image object recognition on the first captured image and the second captured image respectively, the method further includes: According to the application scenario setting, the neural network model is selected from a plurality of preset neural network models, wherein the plurality of preset neural network models include an object recognition model or a face recognition model.

3. The video conference switching method according to claim 1, characterized in that: The step of switching the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image includes: In response to the neural network model detecting that the second captured image includes a specific object, the video source of the video conference is switched to the second camera device.

4. The video conference switching method according to claim 3, characterized in that: The step of switching the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image includes: In response to the neural network model detecting that the second captured image does not include the specific object, the video source of the video conference is switched to the first camera device.

5. The video conference switching method according to claim 1, characterized in that: The step of switching the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image includes: In response to the neural network model detecting that the facial angle of the facial object in the second captured image meets the angle condition, the video source of the video conference is switched to the second camera device.

6. The video conference switching method according to claim 1, characterized in that: The second camera device includes a motion sensor, and the step of switching the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image includes: Determining whether the motion state of the second camera device conforms to a specific motion according to the sensing data of the motion sensor; and In response to the movement state of the second camera device being consistent with the specific action and the neural network model detecting that the second captured image includes a facial object, the video source of the video conference is switched to the second camera device.

7. The video conference switching method according to claim 1, characterized in that: The video conferencing system includes a microphone device, and the method further includes: receiving a sound signal via the microphone device; and Detecting the sound characteristics of the sound signal by another neural network model; and According to the sound characteristics, a video source of the video conference is switched to the first camera device or the second camera device.

8. The video conference switching method according to claim 1, characterized in that: The step of switching the video source of the video conference to the first camera device or the second camera device according to the model recognition result of the first captured image and the model recognition result of the second captured image includes: The video source of the video conference is switched to the first camera device or the second camera device by controlling the output data stream of the media foundation conversion of the device.

9. A video conferencing system, characterized in that: include: First camera device: a second camera device; A storage device storing a plurality of instructions; A processor, coupled to the first camera device, the second camera device and the storage device, executes the plurality of instructions to: Acquire a first image captured by the first camera device and a second image captured by the second camera device; Using a neural network model to perform image object recognition on the first captured image and the second captured image respectively; Switching a video source of the video conference to the first camera device or the second camera device according to a model recognition result of the first captured image and a model recognition result of the second captured image; as well as The video conference is conducted based on the image of the video source.

10. The video conferencing system according to claim 9, characterized in that: The storage device records a plurality of preset neural network models, and the processor executes the plurality of instructions to: According to the application scenario setting, the neural network model is selected from the multiple preset neural network models, wherein the multiple preset neural network models include an object recognition model or a face recognition model.

11. The video conferencing system according to claim 9, characterized in that: The processor executes the plurality of instructions to: In response to the neural network model detecting that the second captured image includes a specific object, the video source of the video conference is switched to the second camera device.

12. The video conferencing system according to claim 11, characterized in that: The processor executes the plurality of instructions to: In response to the neural network model detecting that the second captured image does not include the specific object, the video source of the video conference is switched to the first camera device.

13. The video conferencing system according to claim 9, characterized in that: The processor executes the plurality of instructions to: In response to the neural network model detecting that the facial angle of the facial object in the second captured image meets the angle condition, the video source of the video conference is switched to the second camera device.

14. The video conferencing system according to claim 9, characterized in that: The second camera device includes a motion sensor, and the processor executes the plurality of instructions to: Determining whether the motion state of the second camera device conforms to a specific motion according to the sensing data of the motion sensor; and In response to the movement state of the second camera device being consistent with the specific action and the neural network model detecting that the second captured image includes a facial object, the video source of the video conference is switched to the second camera device.

15. The video conferencing system according to claim 9, characterized in that: Also included is a microphone device coupled to the processor, and the processor executes the plurality of instructions to: receiving a sound signal through the microphone device; and Detecting the sound characteristics of the sound signal by another neural network model; and According to the sound characteristics, a video source of the video conference is switched to the first camera device or the second camera device.

16. The video conferencing system according to claim 9, characterized in that: The processor executes the plurality of instructions to: The video source of the video conference is switched to the first camera device or the second camera device by controlling the output data stream of the media foundation conversion of the device.

Images