Multi-camera video conference image processing system and method

Abstract

The present invention provides a multi-camera video conference image processing system and method, the multi-camera video conference image processing system comprising: a first panoramic camera erected substantially in the middle of a conference room for capturing body characteristics of participants around the scene to generate a first panoramic image; the second camera is erected on the front side of the meeting room and used for capturing the body characteristics of the participants to generate a second image; the single-chip system is used for receiving the first panoramic image and the second image; processing the first panoramic image to generate a panoramic picture of the participant; processing the second image to generate a photographic picture of the participant; corresponding the panoramic picture of each participant to a photographic picture; selecting a panoramic picture or a photographic picture of each participant by taking the body characteristics as a reference; and processing the selected pictures of each participant to generate and output a video picture.

Description

Technical Field

[0001] The present invention relates to a multi-camera video conference image processing method and system, and in particular to a multi-camera video conference image processing method and system capable of obtaining body features. Background Art

[0002] In recent years, panoramic cameras have been widely used in indoor 360-degree conference panoramic photography and recording occasions. For example, panoramic cameras with multiple lenses and fisheye lenses can obtain 360-degree panoramic conference images or images. However, since existing panoramic cameras are generally set up in the middle of the conference room, the viewing angle of the panoramic camera is limited, resulting in the side or back of the head of the object being viewed when the final image is output. This is a problem that currently occurs when panoramic cameras are used in video conferencing.

[0003] Figure 1 FIG. 2 is a schematic diagram of a conventional application of a panoramic camera 20 in a conference room 10 . Figure 2 FIG. 2 is a schematic diagram of the expansion of the panoramic image captured by the panoramic camera 20. Figure 1 In the video conference, the directions of the faces of participants D1, D2, D3, D4, and D5 may change depending on what they are doing. For example, the faces of participants D1, D3, D4, and D5 are all facing the direction of screen 30 and are watching the content displayed on screen 30. Screen 30 may be located at the front side of conference room 10 and in front of conference table 40, and is mainly used to display the content of the video conference. However, for example, the face of participant D2 may be facing the direction of conference table 40. Figure 2 In the example, when the panoramic image captured by the panoramic camera 20 is unfolded, the images displayed by the participants D1, D4, and D5 are the side faces and the back of the heads due to the installation position of the panoramic camera 20. Therefore, this will limit the remote participants from observing the facial expressions or emotions of the participants D1, D4, and D5, and will also affect the remote participants from clearly understanding the content that the participants D1, D4, and D5 want to communicate.

[0004] Therefore, the inventor of this case came up with the present invention after observing the above-mentioned deficiencies. Summary of the invention

[0005] The object of the present invention is to provide a multi-camera video conferencing image processing system for processing images of multiple participants in a conference room. The multi-camera video conferencing image processing system includes: a first panoramic camera, mounted at a roughly middle position of the conference room, for panoramically capturing the physical features of the participants to generate a first panoramic image; at least one second camera, mounted at the front side of the conference room, for capturing the physical features of the participants to generate at least one second image; and a single-chip system for: receiving the first panoramic image and the at least one second image; processing the first panoramic image to generate a panoramic picture of each participant; processing the at least one second image to generate a photographic picture of each participant; corresponding the panoramic picture of each participant to the photographic picture; selecting the panoramic picture or the photographic picture of each participant based on the physical features; and processing the selected picture of each participant to generate and output a video picture.

[0006] Preferably, the multi-camera video conferencing image processing system further includes: an operating device, including at least one of a computer, a tablet computer and a mobile phone.

[0007] Preferably, the single chip system is disposed in at least one of the first panoramic camera and the at least one second camera.

[0008] Preferably, at least one of the first panoramic camera and the at least one second camera includes a PTZ (Pan / Tilt / Zoom) function.

[0009] Preferably, the body feature includes at least one of a facial feature and a limb feature.

[0010] Preferably, the single-chip system selects the panoramic picture or the photographic picture based on the integrity of at least one of the facial feature and the limb feature.

[0011] Preferably, the single-chip system maps the panoramic image of each participant to the photographic image based on the body features in the panoramic image and the photographic image.

[0012] Preferably, the single chip system maps the panoramic image of each participant to the photographic image according to the relative position of the first panoramic camera and the at least one second camera.

[0013] In addition, another object of the present invention is to provide a multi-camera video conferencing image processing method for processing multiple participant images in a conference room, the multi-camera video conferencing image processing method comprising: using a first panoramic camera set up at a roughly middle position of the conference room to panoramically capture the physical features of the participants to generate a first panoramic image; using at least one second camera set up on the front side of the conference room to capture the physical features of the participants to generate at least one second image; processing the first panoramic image to generate a panoramic picture of each participant; processing the at least one second image to generate a photographic picture of each participant; corresponding the panoramic picture of each participant to the photographic picture; selecting the panoramic picture or the photographic picture of each participant based on the physical features; and processing the selected pictures of each participant to generate and output a video picture.

[0014] In order to make those skilled in the art understand the purpose, features and effects of the present invention, the present invention is described in detail through the following specific embodiments and the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of applying a panoramic camera to an existing conference room;

[0016] Figure 2 for Figure 1 A schematic diagram of an unfolded panoramic image captured by a panoramic camera;

[0017] Figure 3 is a block diagram of a multi-camera video conferencing image processing system according to a first embodiment of the present invention;

[0018] Figure 4 is a block diagram of a multi-camera video conferencing image processing system according to a second embodiment of the present invention;

[0019] Figure 5 It is a schematic diagram of a multi-camera video conferencing image processing system set up in a conference room according to an embodiment of the present invention;

[0020] Figure 6 is a schematic diagram of an image captured by a first panoramic camera according to an embodiment of the present invention;

[0021] Figure 7 is a schematic diagram of an image captured by a second camera according to an embodiment of the present invention;

[0022] Figure 8 Based on Figure 6 , Figure 7 A schematic diagram of a first example of selecting a panoramic picture or a photographic picture of each participant from images captured by the first panoramic camera and the second camera;

[0023] Fig. 9 is a schematic diagram of another image captured by the first panoramic camera according to an embodiment of the present invention;

[0024] Fig.10 Based on Fig. 9 , Figure 7 A schematic diagram of a second example of selecting a panoramic picture or a photographic picture of each participant from images captured by the first panoramic camera and the second camera; and

[0025] Fig.11 The figure is a flowchart of a multi-camera video conferencing image processing method according to an embodiment of the present invention.

[0026] Description of Reference Numerals

[0027] 10: Meeting Room

[0028] 20: Panoramic Camera

[0029] 30: Screen

[0030] 100:Multi-camera video conferencing image processing system

[0031] 200: The first panoramic camera

[0032] 202,302: Single chip system

[0033] 204: Panoramic lens module

[0034] 206,306: Wireless module

[0035] 208,308: Universal Serial Bus Port

[0036] 210,310: Microphone

[0037] 212,312: Speakers

[0038] 214,314: Network interface

[0039] 216,316: Display interface

[0040] 218,318: Memory storage module

[0041] 300: Second Camera

[0042] 304: Lens module

[0043] 400:Multi-camera video conferencing image processing method

[0044] S402~S412: Steps

[0045] 500,502: Video screen

[0046] D1~D5: Participants DETAILED DESCRIPTION

[0047] The inventive concept will now be more fully described below with reference to the accompanying drawings in which exemplary embodiments of the inventive concept are shown. The advantages and features of the inventive concept and the methods for achieving them will become apparent below by referring to the exemplary embodiments described in more detail with reference to the accompanying drawings. However, it should be noted that the inventive concept is not limited to the following exemplary embodiments, but can be implemented in various forms. Therefore, the exemplary embodiments are provided only to disclose the inventive concept and to make those skilled in the art aware of the categories of the inventive concept. In the accompanying drawings, the exemplary embodiments of the inventive concept are not limited to the specific examples provided herein and are exaggerated for clarity.

[0048] The terms used herein are only used to illustrate specific embodiments and are not intended to limit the present invention. Unless the context clearly indicates otherwise, the singular terms "a", "an" and "the" used herein are intended to include plural forms as well. The term "and / or" used herein includes any and all combinations of one or more of the relevant listed items. It should be understood that when an element is said to be "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element or there may be an intermediate element.

[0049] Similarly, it should be understood that when an element (such as a layer, region, or substrate) is said to be "on" another element, the element may be directly on the other element, or there may be intervening elements. In contrast, the term "directly" means that there are no intervening elements. It should be further understood that when the terms "include" and "comprising" are used herein, it indicates the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] In addition, the exemplary embodiments in the detailed description will be described by idealized exemplary figures as the inventive concept. Accordingly, the shapes of the exemplary figures may be modified according to manufacturing techniques and / or tolerable errors. Therefore, the exemplary embodiments of the inventive concept are not limited to the specific shapes shown in the exemplary figures, but may include other shapes that may be produced according to the manufacturing process. The regions illustrated in the drawings have general characteristics and are used to illustrate the specific shapes of the elements. Therefore, this should not be considered as being limited to the scope of the inventive concept.

[0051] It should also be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish between various elements. Therefore, the first element in some embodiments may be referred to as the second element in other embodiments without departing from the teachings of the present invention. The exemplary embodiments of the aspects of the inventive concepts explained and illustrated herein include their complementary counterparts. Throughout this specification, the same reference numerals or the same indicators represent the same elements.

[0052] Furthermore, exemplary embodiments are described herein with reference to cross-sectional views and / or plan views, which are idealized exemplary illustrations. Therefore, deviations from the illustrated shapes due to, for example, manufacturing techniques and / or tolerances are expected. Therefore, exemplary embodiments should not be considered limited to the shapes of the regions shown herein, but are intended to include deviations in shapes due to, for example, manufacturing. Therefore, the regions shown in the figures are schematic, and their shapes are not intended to illustrate the actual shape of the regions of the device, nor are they intended to limit the scope of the exemplary embodiments.

[0053] In addition, it should be understood that the image captured by the camera will include other background images, such as other physical features of the participants, the background of the conference room, items on the conference table, etc. In addition, the distance between the faces of the participants will also change due to the location of the participants, and the size ratio of the faces of the participants in the image is also related to the distance between the participants and the camera. Therefore, it should be understood by those skilled in the art that the images captured by the camera shown in the drawings of the present invention are only exemplary and are for the purpose of ease of description and understanding.

[0054] See also Figure 3 As shown, Figure 3 FIG. 1 is a block diagram of a multi-camera video conferencing image processing system 100 according to a first embodiment of the present invention. Figure 3 As shown, the multi-camera video conferencing image processing system 100 according to the present invention includes: a first panoramic camera 200, used for panoramically capturing the body features of the participants to generate a first panoramic image; and a second camera 300, used for capturing the body features of the participants to generate a second image.

[0055] Specifically, refer to Figure 3The first panoramic camera 200 includes: a single chip system 202 for executing the functions of the first panoramic camera 200 and performing image processing. Specifically, the single chip system 202 includes, for example, Rockchip's RK3588 chip or Qualcomm's QCS8250 chip; a panoramic lens module 204 for panoramically capturing the physical features of the participants, including a lens, an image sensor, and a motor (for example, a voice coil motor or a PTZ motor); a wireless module 206 for wirelessly connecting to other devices. For example, the wireless module 206 may include a wireless network or Bluetooth to connect to the second camera 300 or other devices; a universal serial bus (USB) port 208 for wired connection to the second camera 300 or other devices; a microphone 210 for receiving sound; a speaker 212 for outputting sound; a network interface 214 for connecting to a network; a display interface 216 for displaying the functional interface of the first panoramic camera 200 for easy operation; and a memory storage module 218 for storing data, for example, processed images or image recognition related data. However, the present invention is not limited thereto, and the first panoramic camera 200 may further include other functions and components, such as automatic exposure processing (Auto Exposure), white balance (Auto White Balance), professional image adjustment and support for low-light photography capabilities to adapt to the conference room environment, and can still provide the best quality images even when the lighting in the conference room is lowered.

[0056] Specifically, refer to Figure 3 , the second camera 300 includes: a single chip system 302 for executing the functions of the second camera 300 and performing image processing, specifically, the single chip system 302 includes, for example, a SAV636G chip of SigmaStar or a NT98530 chip of Novatek, etc.; a lens module 304 for capturing the physical characteristics of the participants, including a lens, an image sensor, and a motor (for example, a voice coil motor or a PTZ motor), etc.; a wireless module 306 for wirelessly connecting to other devices, for example, the wireless module 306 may include a wireless network or Bluetooth to connect to the first panoramic camera 200 or other devices; and a USB port 308 for wired connection to the first panoramic camera 200 or other devices. However, the present invention is not limited thereto, such as the first panoramic camera 200, the second camera 300 may further include other functions and elements.

[0057] See also Figure 4 As shown, Figure 4 FIG. 1 is a block diagram of a multi-camera video conferencing image processing system 100 according to a second embodiment of the present invention. Figure 4 The second embodiment shown is Figure 3The difference of the first embodiment shown is that the elements performing other functions other than the photography and data transmission related functions of the first panoramic camera 200 are located in the second camera 300 .

[0058] Specifically, refer to Figure 4 , the first panoramic camera 200 includes: a single chip system 202, for executing the functions of the first panoramic camera 200 and performing image processing, specifically, the single chip system 202 includes, for example, Rockchip's RK3588 chip or Qualcomm's QCS8250 chip, etc.; a panoramic lens module 204, for panoramically capturing the physical features of the participants, including a lens, an image sensor, and a motor (e.g., a voice coil motor or a PTZ motor), etc.; a wireless module 206, for wirelessly connecting to other devices, for example, the wireless module 206 may include a wireless network or Bluetooth to connect to the second camera 300 or other devices; and a universal serial bus (USB) port 208, for wired connection to the second camera 300 or other devices. However, the present invention is not limited thereto, and the first panoramic camera 200 may further include other functions and elements.

[0059] Specifically, refer to Figure 4 , the second camera 300 includes: a single-chip system 302 for executing the functions of the second camera 300 and performing image processing. Specifically, the single-chip system 302 includes, for example, a SAV636G chip of SigmaStar or a NT98530 chip of Novatek; a lens module 304 for capturing the physical characteristics of the participants, including a lens, an image sensor, and a motor (for example, a voice coil motor or a PTZ motor); a wireless module 306 for wirelessly connecting to other devices. For example, the wireless module 306 may include a wireless network or Bluetooth to connect to the first panoramic camera 200 or other devices; a USB port 308 for wired connection to the first panoramic camera 200 or other devices; a microphone 310 for receiving sound; a speaker 312 for outputting sound; a network interface 314 for connecting to a network; a display interface 316 for displaying the functional interface of the first panoramic camera 200 for easy operation; and a memory storage module 318 for storing data, for example, processed images or related data of image recognition. However, the present invention is not limited thereto, and the second camera 300 may further include other functions and elements.

[0060] It is understood that the present invention is not limited to the above-mentioned embodiments, and the first panoramic camera 200 and the second camera 300 may also include components such as microphones, speakers, network interfaces, display interfaces, and memory storage modules to operate according to user settings. For example, the microphone of the first panoramic camera 200 is used to receive sound, and the speakers of the first panoramic camera 200 and the second camera 300 are used to play sound.

[0061] In addition, the multi-camera video conference image processing system 100 may further include an operating device, which includes at least one of a computer, a tablet computer, and a mobile phone to operate the functions of the first panoramic camera 200 and the second camera 300. For example, the user can select the image output mode, set the focus object among the participants, select the image zoom size, etc. in the software installed in the operating device.

[0062] It should be noted that the first panoramic camera 200 and the second camera 300 include a single chip system 202 and a single chip system 302, respectively. Therefore, the first panoramic camera 200 and the second camera 300 can use their own single chip systems and can be directly used for video conferencing with a remote device without being connected to other operating devices. The first panoramic camera 200 and the second camera 300 can also share a single chip system, without the need for both to be equipped with their own single chip systems, so as to reduce module costs.

[0063] According to the embodiments of the present case, the single-chip system 202 and the single-chip system 302 can be respectively used to: receive a first panoramic image and a second image; process the first panoramic image to generate a panoramic picture of each participant; process the second image to generate a photographic picture of each participant; correspond the panoramic picture of each participant to the photographic picture; select the panoramic picture or the photographic picture of each participant based on body features; and process the selected picture of each participant to generate and output a video picture.

[0064] See also Figure 5 shown. Figure 5 Schematic diagram of a multi-camera video conferencing image processing system according to an embodiment of the present invention being set up in a conference room. Generally, when the multi-camera video conferencing image processing system 100 is set up in a conference room 10, the first panoramic camera 200 is set up at a substantially middle position of the conference room 10, or is set up among the participants and is surrounded by the participants, and the second camera 300 is set up at the front side of the conference room 10. For example, the second camera 300 can be set up on the upper side of the screen 30, near the screen 30, or the second camera 300 can be integrated into the screen 30. However, the present invention is not limited thereto, and the position of the second camera 300 can be arbitrarily changed or multiple second cameras can be set up according to user needs.

[0065] In addition, from Figure 5 It can be seen that the faces of participants D1, D3, D4, and D5 are all facing the screen 30, while the face of participant D2 is facing the conference table 40. Therefore, it will be described below how the present invention overcomes the inconsistency of the faces of the participants and solves the problem that the images displayed for the participants are the side faces and the back of the heads.

[0066] See also Figure 6 , Figure 7 shown. Figure 6 is a schematic diagram of an image captured by a first panoramic camera according to an embodiment of the present invention; Figure 7 FIG. 1 is a schematic diagram of an image captured by a second camera according to an embodiment of the present invention. Figure 6 In the image obtained by unfolding the image captured by the first panoramic camera 200, according to the installation position of the first panoramic camera 200 and the face orientations of the participants D1-D5, the panoramic image of participant D1 is displayed as the right rear side of the face, the panoramic images of participants D2 and D3 are displayed as the front of the face, the panoramic image of participant D4 is displayed as the left side of the face, and the panoramic image of participant D5 is displayed as the left rear side of the face. Figure 7 In the image captured by the second camera 300, according to the installation position of the second camera 300 and the face orientation of the participants D1-D5, the photographic images of the participants D1, D3, D4, and D5 are displayed as the front of the face, while the photographic image of the participant D2 is displayed as the left side of the face. Therefore, from the panoramic image and the photographic image of the participants, since the panoramic image of the participants D1, D4, and D5 is not displayed as the front of the face, while the photographic image of the participants D1, D4, and D5 is displayed as the front of the face, the present invention will replace the panoramic image of the participants D1, D4, and D5 with the photographic image of the participants D1, D4, and D5, so that all the participants can have the image displayed as the front of the face on the output video screen.

[0067] It is understandable that, in the present invention, the physical features of the participants may be at least one of body features and facial features. For example, the body features may be set as the body features of the participants. When the body features are set as the body features of the participants, the first panoramic camera 200 and the second camera 300 will capture the body features of the participants as completely as possible, so that the participants connected remotely can clearly see the body movements of each participant. For another example, the body features may be set as the facial features of the participants. When the body features are set as the facial features of the participants, the first panoramic camera 200 and the second camera 300 will capture the facial features of the participants as completely as possible, so that the participants connected remotely can clearly see the facial expressions of each participant.

[0068] In addition, during the image processing process, in addition to the above-mentioned step of selecting a photographic frame or a panoramic frame, the single-chip system can also further perform an image processing process. For example, a computer vision algorithm is used to process the incoming frame or image, such as edge detection, image segmentation, and feature extraction. Again, a person detection algorithm is used to identify and locate the person in the frame or image, such as the face, body shape, or other target of interest of the attendee. Again, a person detection algorithm is used to identify and locate the person in the frame or image. Again, once the person is identified, the artificial intelligence algorithm can use motion tracking technology to track the person's movement over time. This allows the camera to adjust its position or field of view (FOV) and framing to keep the person in the center of the frame. Again, the artificial intelligence algorithm can also extract features such as facial features or body shape, which can be used to trigger specific actions or commands. Again, the decision algorithm can determine the best course of action based on the information collected. For example, if the attendee is talking, it may be decided to zoom in on the attendee's face, or if the attendee is walking around, switch to a wider angle view. However, the present invention is not limited to this, and various image processing functions can be arbitrarily changed or increased or decreased according to user needs.

[0069] Specifically, when selecting a photographic picture or a panoramic picture of a participant, the panoramic picture or the photographic picture can be selected based on the integrity of the facial features. For example, the above or any existing algorithm can be used to determine the integrity of facial features. For example, the facial integrity of both the panoramic picture and the photographic picture can be determined based on the integrity of the participant's facial features. That is, a picture with higher integrity of the facial features is selected. For example, the angle of the face can be determined based on the existing algorithm. If the facial angle is forward, it can be determined to be 100% complete. If the facial angle is to the left or right, it can be determined to be 50% complete. If the facial angle is to the back, it can be determined to be 0% complete. And so on. In addition, if the integrity of the two is the same, for example, the panoramic picture or the photographic picture of participant D3 are both frontal, the panoramic picture of the first panoramic camera 200 can be used as an option so that the second camera 300 can be used to capture the pictures of other participants. For another example, the symmetry of facial features can also be used to judge completeness. A panoramic picture or photographic picture with higher symmetry can be judged as having more complete facial features and be regarded as a front view; a panoramic picture or photographic picture with lower symmetry can be judged as having less complete facial features and be regarded as a side view.

[0070] Alternatively, the panoramic picture or photographic picture can be selected based on the integrity of the limb features. For example, the above or any existing algorithm can be used to determine the integrity of the limb features. For example, a panoramic picture or photographic picture with two arms can be regarded as the front and can be determined to be 100% complete. A panoramic picture or photographic picture with only one arm, which is the left or right side, can be determined to be 50% complete. In addition, facial features can be further used to assist in judging the completeness. For example, in a panoramic picture or photographic picture with two arms, the front and back can be distinguished with the help of facial features. For another example, the body width can also be used to judge the completeness. A panoramic picture or photographic picture with a larger body width has a higher integrity and can be regarded as the front; a panoramic picture or photographic picture with a smaller body width has a lower integrity and can be regarded as the side.

[0071] See also Figure 8 shown. Figure 8 Based on Figure 6 , Figure 7 A schematic diagram of a first example of selecting a panoramic picture or a photographic picture of each participant from images captured by the first panoramic camera and the second camera.

[0072] Specifically, after selecting the panoramic images or photographic images of each participant, the multi-camera video conferencing image processing system 100 will process and arrange the images for use in the video conference. Figure 8 According to the above-mentioned screen selection method, the multi-camera video conference image processing system 100 can output the video screen 500. Specifically, the multi-camera video conference image processing system 100 selects the photographic screens of the participants D1, D4, and D5 and the panoramic screens of the participants D2 and D3 (indicated by the dotted boxes) from the panoramic screens and photographic screens of the participants, based on the fact that the panoramic screens of the participants D1, D4, and D5 do not show the front face and the photographic screens of the participants D1, D4, and D5 show the front face, and arranges these screens into the video screen 500 according to the user's selection method.

[0073] In addition, it is understandable that the video images output by the multi-camera video conferencing image processing system 100 are not limited to Figure 8 For example, the multi-camera video conferencing image processing system 100 may include any image processing and editing methods for user selection.

[0074] See also Fig. 9 . Fig. 92 is a schematic diagram of another image captured by the first panoramic camera according to an embodiment of the present invention. According to an embodiment of the present invention, since the installation position of the first panoramic camera 200 only needs to be located approximately in the middle of the conference room 10 and does not need to be placed at any specific angle (for example, the first panoramic camera 200 is rotated to face forward), the image captured by the first panoramic camera 200 may be as follows: Fig. 9 shown. Fig. 9 The display is based on the different starting angles of the first panoramic camera 200, resulting in the order D4-D5-D1-D2-D3 of the participants D1-D5 displayed in the expanded view being different from that in the expanded view. Figure 6 In this case, the panoramic images and the photographic images of the participants D1-D5 need to be matched with each other before the panoramic image or the photographic image can be selected.

[0075] Specifically, in the present invention, the panoramic picture of each participant can be matched to the photographic picture based on the body features in the panoramic picture and the photographic picture. For example, feature difference calculation can be used to match the panoramic picture of each participant to the photographic picture. Specifically, the feature difference calculation is mainly to identify the differences or similarities of the participants based on the feature differences of the participants. For example, the correspondence can be established by comparing the descriptors between the features or keypoints. This can be done using various matching methods, such as nearest neighbor matching, random sampling consistent matching, or deep learning-based matching.

[0076] Alternatively, in the present invention, the panoramic images of each participant can be matched to the photographic images according to the relative positions of the first panoramic camera 200 and the second camera 300. For example, before capturing the images, the first panoramic camera 200 and the second camera 300 can confirm each other's positions by capturing images, wireless connection or any other means, so that the first panoramic camera 200 unfolds the panoramic image along the direction of the second camera 300. Accordingly, in this case, the panoramic image of the first panoramic camera 200 will correspond to the photographic image of the second camera 300, so that the participants are completely matched.

[0077] See also Fig.10 shown. Fig.10 Based on Fig. 9 , Figure 7 A schematic diagram of a second example of selecting and outputting a panoramic picture or a photographic picture of each participant from images captured by the first panoramic camera and the second camera.

[0078] Similar to Figure 8 ,exist Fig.10 In the multi-camera video conferencing image processing system 100, a video screen 502 can be output. According to the foregoing, after the panoramic screen of the first panoramic camera 200 is completely matched to the photographic screen of the second camera 300 so as to completely correspond between the participants, the multi-camera video conferencing image processing system 100 can select the photographic screens of the participants D1, D4, and D5 and the panoramic screens of the participants D2 and D3 (indicated by the dotted boxes), and arrange these screens into a video screen 502 according to the user's selection method. For example, the participants D2, D3, and D4 are the focus objects pre-set by the user. Therefore, in the video screen 502, the image displayed above the video screen 502 is the image obtained by expanding the image captured by the first panoramic camera 200, that is, completely corresponding. Fig. 9 In addition, the selected images of the focus objects D2, D3, and D4 are enlarged and displayed below the video screen 502. It is understood that the selected images can be set in any selected and arranged manner. Fig.10 video screen 502. For example, the selected screen may be displayed above the video screen 502, that is, a screen showing the front faces of the participants D1 to D5. In addition, the focus object may be selected in any existing manner, such as pre-setting the focus object, setting the person who is speaking as the focus object, or setting the participant at a specific position as the focus object. For another example, participants D2, D3, and D4 related to the topic being discussed are set as the focus objects, and the person D4 who is speaking is set as a special focus object. In addition to the screens of all the participants, the corresponding screens of the focus objects D2 and D3 may be enlarged and displayed in the video screen, and the corresponding screen of the special focus object D4 may be enlarged and displayed in a close-up. In addition, the focus objects are not limited to three, and the arrangement method is not limited to the above method, and can be adjusted as needed.

[0079] In addition, it is understandable that the video images output by the multi-camera video conferencing image processing system 100 are not limited to Figure 8 , Fig.10 For example, the multi-camera video conferencing image processing system 100 may include any existing image processing and editing methods for user selection.

[0080] See also Fig.11 . Fig.11The flowchart of the multi-camera video conferencing image processing method according to an embodiment of the present invention. In the present invention, the multi-camera video conferencing image processing method 400 of the embodiment of the present invention includes: step S402, capturing a first panoramic image through a first panoramic camera and capturing a second image through a second camera; step S404, processing the first panoramic image to generate a panoramic picture of each participant; step S406, processing the second image to generate a photographic picture of each participant; step S408, determining whether the panoramic picture of each participant corresponds to the photographic picture; step S410, corresponding the panoramic picture of each participant to the photographic picture; and step S412, selecting the panoramic picture or the photographic picture of each participant based on the body characteristics.

[0081] It is understandable that in step S408, if it is determined that the panoramic image of each participant does not correspond to the photographic image, step S410 is executed. That is, the panoramic image of each participant is matched to the photographic image based on the body features in the panoramic image and the photographic image or based on the relative position of the first panoramic camera and the second camera. After confirming that the panoramic image of each participant should correspond to the photographic image in step S408 or step S410, step S412 is executed.

[0082] In addition, it is understandable that, in the present invention, multiple second cameras 300 can be set. When multiple second cameras 300 are set, the multi-camera video conferencing image processing system 100 of the present invention also uses the above-mentioned method to process images and pictures. For example, when two second cameras 300 are set, the two second cameras 300 capture two second images. When processing the two second images, two groups of photographic pictures of each participant are generated. Therefore, when selecting the panoramic picture or photographic picture of each participant, the picture with the highest integrity will be selected for each participant from one panoramic picture and two photographic pictures. Since the two second cameras 300 are set at two different positions, the participants can be photographed from different angles. Therefore, when multiple second cameras 300 are set, it is easier to obtain a participant picture with higher integrity.

[0083] The above describes the implementation mode of the present invention through specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention; any other equivalent changes or modifications that do not depart from the spirit disclosed by the present invention should be included in the following claims.

Claims

1. A multi-camera video conference image processing system, used for processing images of multiple participants in a conference room, the multi-camera video conference image processing system comprising: A first panoramic camera is installed at a substantially middle position of the conference room and is used to capture the body features of the multiple participants in a panoramic manner to generate a first panoramic image; At least one second camera, mounted at the front side of the conference room, for capturing the physical features of the plurality of participants to generate at least one second image; as well as Single chip system for: receiving the first panoramic image and the at least one second image; Processing the first panoramic image to generate a panoramic picture of each participant; Processing the at least one second image to generate a photographic image of each participant; Matching the panoramic images of each participant to the photographic image; Based on the physical features, selecting the panoramic picture or the photographic picture of each participant; as well as The images of each selected participant are processed to generate and output video images.

2. The multi-camera video conferencing image processing system according to claim 1, further comprising: The operating device includes at least one of a computer, a tablet computer and a mobile phone.

3. The multi-camera video conferencing image processing system according to claim 1, wherein: The single chip system is disposed in at least one of the first panoramic camera and the at least one second camera.

4. The multi-camera video conferencing image processing system according to claim 1, wherein: At least one of the first panoramic camera and the at least one second camera includes a PTZ function.

5. The multi-camera video conferencing image processing system according to claim 1, wherein: The body feature includes at least one of a facial feature and a limb feature.

6. The multi-camera video conferencing image processing system according to claim 5, wherein: The single-chip system selects the panoramic picture or the photographic picture based on the integrity of at least one of the facial features and the limb features.

7. The multi-camera video conferencing image processing system according to claim 1, wherein: The single-chip system matches the panoramic image of each participant to the photographic image based on the body features in the panoramic image and the photographic image.

8. The multi-camera video conferencing image processing system according to claim 1, wherein: The single chip system corresponds the panoramic picture of each participant to the photographic picture according to the relative position of the first panoramic camera and the at least one second camera.

9. A multi-camera video conference image processing method for processing multiple participant images in a conference room, the multi-camera video conference image processing method comprising: Capturing the body features of the plurality of participants in a panoramic manner by using a first panoramic camera installed at a substantially middle position of the conference room to generate a first panoramic image; Capturing the physical features of the plurality of participants through at least one second camera mounted on the front side of the conference room to generate at least one second image; Processing the first panoramic image to generate a panoramic picture of each participant; Processing the at least one second image to generate a photographic image of each participant; Matching the panoramic images of each participant to the photographic image; Based on the physical features, selecting the panoramic picture or the photographic picture of each participant; as well as The images of each selected participant are processed to generate and output video images.

10. The multi-camera video conferencing image processing method according to claim 9, wherein: The body feature includes at least one of a facial feature and a limb feature.

11. The multi-camera video conferencing image processing method according to claim 10, wherein: The panoramic picture or the photographic picture is selected based on the completeness of at least one of the facial features and the limb features.

12. The multi-camera video conferencing image processing method according to claim 9, wherein: The panoramic image of each participant is matched to the photographic image according to the body features in the panoramic image and the photographic image.

13. The multi-camera video conferencing image processing method according to claim 9, wherein: The panoramic picture of each participant is matched to the photographic picture according to the relative position of the first panoramic camera and the at least one second camera.

Images