Surround View Conferencing System and Method

Abstract

A surround-view video conferencing system and method are disclosed. The surround-view video conferencing system includes a surround-view image generating device and a control device. The control device analyzes a surround-view image to identify multiple image objects within it. Based on a privacy mode, the control device selects an image object to be removed from the multiple image objects. The control device then removes an image corresponding to the image object to be removed from the surround-view image. Based on the surround-view image and a background-filling image, the control device generates a surround-view conferencing image. The surround-view video conferencing technology provided by this invention allows for the selective hiding of some image objects, improving the service and quality of online meetings.

Description

Technical Field

[0001] This invention relates to a panoramic video conferencing system and method. More specifically, this invention relates to a panoramic video conferencing system and method that provides multiple functions. Background Technology

[0002] In recent years, due to the increasing demand for remote work and online meetings, many different models and services have emerged in the market that differ from general video conferencing. One of these is panoramic video conferencing, which can provide simultaneous video for multiple people.

[0003] However, compared to regular video conferencing, multi-person meetings using panoramic imaging typically face more technical challenges, such as the presentation and smoothness of the panoramic images during the meeting. Furthermore, because panoramic imaging covers a wider field of view, it may capture sensitive objects or participants unrelated to the meeting, raising concerns about corporate privacy.

[0004] In view of this, how to provide a panoramic video conferencing technology that can take into account privacy and provide multiple functions in panoramic video conferencing to improve the service quality of panoramic video conferencing is an urgent goal for the industry. Summary of the Invention

[0005] One object of the present invention is to provide a panoramic video conferencing system. The panoramic video conferencing system includes a panoramic video generating device and a control device connected to the panoramic video generating device. The panoramic video generating device generates a panoramic video. The control device analyzes the panoramic video to identify multiple image objects within it. Based on a privacy mode, the control device selects an image object to be removed from the multiple image objects. The control device removes an image corresponding to the image object to be removed from the panoramic video. The control device generates a panoramic conference video based on the panoramic video and a background-filling image.

[0006] Another object of the present invention is to provide a panoramic video conferencing method for a panoramic video conferencing system, the system comprising a panoramic video generating device and a control device. The panoramic video conferencing method is executed by the control device and includes the following steps: analyzing a panoramic video received from the panoramic video generating device to identify a plurality of image objects in the panoramic video; selecting an image object to be removed from the plurality of image objects based on a privacy mode; removing an image corresponding to the image object to be removed from the panoramic video; and generating a panoramic conferencing video based on the panoramic video and a background filling image.

[0007] In one embodiment of the present invention, the control device further stores an image recognition model, and the control device further performs the following operation: based on an edge detection algorithm and the image recognition model, it identifies the plurality of image objects in the panoramic image, wherein the image recognition model is generated based on the features of the plurality of objects and the features of the plurality of meeting participants.

[0008] In one embodiment of the present invention, it further includes: a microphone array connected to the control device via a network communication through at least one of a wired connection and a wireless connection, for sensing sound in a conference room to generate a sound signal; wherein the control device further performs the following operations: receiving the sound signal from the microphone array; determining a source of speech based on the sound signal; comparing multiple meeting participants' characteristics with the panoramic image corresponding to the source of speech to determine whether the panoramic image corresponding to the source of speech contains a first meeting participant characteristic among the multiple meeting participants' characteristics; when it is determined that the panoramic image corresponding to the source of speech contains the first meeting participant characteristic, generating a focused image corresponding to the source of speech; and generating the panoramic conference image based on the focused image and the panoramic image.

[0009] In one embodiment of the present invention, it further includes: a display for playing a remote conference video and a remote conference audio; wherein the control device further performs the following operations: generating an exclusion area based on a position of the display; comparing the speech source and the exclusion area to determine whether the speech source comes from the exclusion area; and when the speech source comes from the exclusion area, not generating the focused image corresponding to the speech source.

[0010] In one embodiment of the present invention, the panoramic image generating device further includes: a first photography device and a second photography device, the first photography device generating a first image, the second photography device generating a second image, and the panoramic image being composed of the first image and the second image.

[0011] In one embodiment of the invention, the control device further performs the following operations: detecting whether a first meeting person with the characteristics of a first meeting person appears in the first image; and when the first meeting person with the characteristics of the first meeting person appears in the first image, generating a close-up image of the first meeting person as the first image.

[0012] In one embodiment of the invention, the control device further performs the following operations: detecting whether a second meeting person with the characteristics of a second meeting person appears in the second image; and when the second meeting person with the characteristics of the second meeting person appears in the second image, generating a close-up image of the second meeting person as the second image.

[0013] In one embodiment of the invention, the control device further performs the following operations: calculating multiple color feature values ​​of multiple neighboring blocks of the image object to be removed; and generating the background filling image based on the multiple color feature values.

[0014] In one embodiment of the present invention, the control device further stores an image recognition model, and the panoramic image conferencing method further includes the following steps: identifying the plurality of image objects in the panoramic image based on an edge detection algorithm and the image recognition model, wherein the image recognition model is generated based on the features of the plurality of objects and the features of the plurality of meeting participants.

[0015] In one embodiment of the present invention, the surround-view conferencing system further includes: a microphone array connected to the control device via at least one of a wired connection and a wireless connection, for sensing sound in a conference room to generate an audio signal; wherein the surround-view video conferencing method further includes the following steps: receiving the audio signal from the microphone array; determining a speaking source based on the audio signal; comparing multiple meeting participant characteristics with the surround-view image corresponding to the speaking source to determine whether the surround-view image corresponding to the speaking source contains a first meeting participant characteristic among the multiple meeting participant characteristics; when it is determined that the surround-view image corresponding to the speaking source contains the first meeting participant characteristic, generating a focused image corresponding to the speaking source; and generating the surround-view conferencing image based on the focused image and the surround-view image.

[0016] In one embodiment of the present invention, the panoramic conferencing system further includes: a display for playing a remote conferencing video and a remote conferencing audio; wherein the panoramic video conferencing method further includes the following steps: generating an exclusion area based on a position of the display; comparing the speech source and the exclusion area to determine whether the speech source comes from the exclusion area; and when the speech source comes from the exclusion area, not generating the focused image corresponding to the speech source.

[0017] In one embodiment of the present invention, the panoramic image generating device further includes: a first photography device and a second photography device, the first photography device generating a first image, the second photography device generating a second image, and the panoramic image being composed of the first image and the second image.

[0018] In one embodiment of the present invention, the panoramic video conferencing method further includes the following steps: detecting whether a first meeting person with the characteristics of a first meeting person appears in the first image; and when the first meeting person with the characteristics of the first meeting person appears in the first image, generating a close-up image of the first meeting person as the first image.

[0019] In one embodiment of the present invention, the panoramic video conferencing method further includes the following steps: detecting whether a second meeting person with the characteristics of a second meeting person appears in the second image; and when the second meeting person with the characteristics of the second meeting person appears in the second image, generating a close-up image of the second meeting person as the second image.

[0020] In one embodiment of the present invention, the panoramic image conferencing method further includes the following steps: calculating multiple color feature values ​​of multiple neighboring blocks of the image object to be removed; and generating the background filling image based on the multiple color feature values.

[0021] The panoramic video conferencing technology (including at least a system and method) provided by this invention analyzes a panoramic image and identifies multiple image objects within it. Furthermore, based on a privacy mode, this panoramic video conferencing technology can select one image object to be removed from the panoramic image. Finally, a panoramic meeting image is generated based on the panoramic image and a background-filled image. This panoramic video conferencing technology, through the aforementioned mechanism, allows for the selective hiding of some image objects, providing enterprises with more flexible privacy options. In addition, this panoramic video conferencing technology offers numerous other features, enhancing the service and quality of online meetings.

[0022] The following detailed description of the technology and embodiments of the present invention, in conjunction with the accompanying drawings, is provided so that those skilled in the art can understand the technical features of the claimed invention. Attached Figure Description

[0023] Figure 1 This is a schematic diagram depicting the architecture of the panoramic video conferencing system according to the first embodiment.

[0024] Figure 2 This is a schematic diagram depicting the control device architecture of the first embodiment;

[0025] Figure 3A It is an image diagram depicting multiple object features of the first embodiment;

[0026] Figure 3B It is an image schematic diagram depicting the removal of an image object according to the first embodiment; and

[0027] Figure 4 This is a partial flowchart depicting the panoramic video conferencing method of the second embodiment.

[0028] [Symbol Explanation]

[0029] 1: Surround View Conferencing System

[0030] 2: Control device

[0031] 3: Surround View Image Generation Device

[0032] 4: Microphone Array

[0033] 5: Monitor

[0034] 301: Surround View Imaging

[0035] 21: Storage

[0036] 23: Send / Receive Interface

[0037] 25: Processor

[0038] 303: Image

[0039] 305: Image

[0040] OB1, OB2: Image objects

[0041] S401, S403, S405, S407: Steps Detailed Implementation

[0042] The following description, through embodiments, explains the panoramic video conferencing system and method provided by the present invention. However, these embodiments are not intended to limit the implementation of the invention to any environment, application, or manner described herein. Therefore, the description of the embodiments is merely illustrative and not intended to limit the scope of the invention. It should be understood that in the following embodiments and drawings, elements not directly related to the present invention have been omitted and are not shown, and the dimensions of each element and the proportions between elements are merely illustrative and not intended to limit the scope of the invention.

[0043] First, the applicable scenarios of this invention will be explained, and its schematic diagram is depicted in... Figure 1 .like Figure 1 As shown, in the first embodiment of the present invention, the panoramic video conferencing system 1 includes at least a panoramic video generating device 3 and a control device 2. In this scenario, the control device 2 can be connected to the panoramic video generating device 3 via a wired or wireless means (e.g., a wireless communication network), and the panoramic video generating device 3 is used to continuously generate panoramic videos 301 corresponding to the meeting (e.g., at a frequency of 30 frames per second). For example, the panoramic video generating device 3 can be placed on a table in the conference room to generate videos of all participants in the meeting.

[0044] In some embodiments, the panoramic video conferencing system 1 further includes a microphone array 4 and a display 5, which can be connected to the control device 2 via wired or wireless means (e.g., a wireless communication network).

[0045] It should be noted that the panoramic image generating device 3 is used to generate a 360-degree panoramic image 301. The panoramic image generating device 3 may consist of at least two photographic devices, and the panoramic image 301 may be composed of multiple image frames (i.e., multiple frames) generated by these photographic devices. For example, the panoramic image generating device 3 may consist of three photographic devices with a field of view (FOV) of 120 degrees. Therefore, the images from each of these photographic devices can be combined to generate a 360-degree panoramic image 301.

[0046] In this invention, the panoramic video conferencing system 1 is a local conferencing system. It is connected to other remote conferencing systems (not shown), and conducts online meetings by transmitting panoramic video feeds to the remote conferencing systems. The following paragraphs will focus on the various functions of the local panoramic video conferencing system 1. Those skilled in the art should understand the implementation details of the connections between different conferencing systems, and therefore will not be elaborated upon.

[0047] In order to improve the quality of online meetings, the panoramic video conferencing system 1 mainly provides at least three types of functions, namely, privacy object hiding function, speech source focusing function, and loop playback function. The following paragraphs will describe in detail the implementation details related to the present invention.

[0048] The control device 2 in the first embodiment of the present invention is shown in the schematic diagram. Figure 2 The control device 2 includes a storage device 21, a transceiver interface 23, and a processor 25, the processor 25 being electrically connected to the storage device 21 and the transceiver interface 23. The storage device 21 may be a memory, a Universal Serial Bus (USB) disk, a hard disk, an optical disk, a USB flash drive, or any other storage medium or circuit known to those skilled in the art and having the same function. The transceiver interface 23 is an interface capable of receiving and transmitting data, or other interfaces capable of receiving and transmitting data known to those skilled in the art. The transceiver interface 23 can receive data from sources such as external devices, external web pages, external applications, etc. The processor 25 may be various processing units, a central processing unit (CPU), a microprocessor, or other computing devices known to those skilled in the art.

[0049] Next, the operation of the first embodiment of the present invention will be described below. Please refer to the following: Figure 1 and Figure 2The first embodiment of the present invention relates to a privacy object hiding function provided by the panoramic video conferencing system 1. It should be noted that during a meeting, due to privacy concerns or meeting requirements, it is necessary to hide some meeting participants, sensitive objects, and clutter in the meeting room. Therefore, in this embodiment, the control device 2 can identify objects in the panoramic image 301 and remove some of the objects (e.g., make the objects transparent), generating a panoramic meeting image after removal.

[0050] Specifically, the control device 2 analyzes the panoramic image 301 to identify multiple image objects within it. When a user selects a privacy mode via the control device 2, the control device 2 can select an image object to be removed from the multiple image objects. In some embodiments, the control device 2 can also select multiple image objects to be removed from the multiple image objects. It should be noted that the privacy mode may include a list of image objects to be removed, indicating the image objects that need to be hidden. For example, this list can be selected from multiple preset templates, or it can be dynamically generated by the user after the control device 2 has identified multiple image objects in the panoramic image 301 (e.g., manually selecting meeting participants, sensitive objects, etc., that need to be hidden).

[0051] In some embodiments, the control device 2 further stores an image recognition model, and the control device 2 identifies image objects in the panoramic image 301 based on an edge detection algorithm and the image recognition model. For example, the control device 2 performs edge detection on the panoramic image 301 using an edge detection algorithm to find objects in the image (i.e., the panoramic image 301). Then, through the image recognition model, the image data of this area is classified and feature value compared to identify the location and identity data of objects and people in the image.

[0052] It should be noted that the image recognition model can be used to identify the multiple image objects in the panoramic image 301. The image recognition model can be generated by training a neural network architecture based on multiple object features and multiple meeting personnel features. Those with ordinary knowledge in the art should be able to understand its operation based on the above description, so it will not be elaborated further.

[0053] Next, in this embodiment, the control device 2 removes an image corresponding to the object to be removed from the panoramic image 301. Finally, the control device 2 generates a panoramic conference image based on the panoramic image and a background-filling image.

[0054] In some embodiments, the control device 2 first calculates multiple color feature values ​​of multiple neighboring blocks of the image object to be removed. Then, based on these multiple color feature values, the control device 2 generates the background-filled image. For example, the control device 2 can separate the foreground object and the background using a nearest neighbor algorithm (e.g., the k-nearest neighbors (KNN) algorithm) and calculate the color feature value distribution of the removed blank area. Then, a background-filled image is generated to fill the blank area. In some embodiments, the control device 2 can also perform a second region-filling operation on the blank area based on a sub-patch texture synthesis algorithm to make the blended image more natural.

[0055] To facilitate understanding, let's take a practical example. Please also refer to... Figure 3A and Figure 3B . Figure 3A An image 303 with multiple object features is illustrated. After analysis of image 303 by control device 2, image objects OB1 and OB2 are identified in image 303. In this example, when both image objects OB1 and OB2 are image objects to be removed, control device 2 removes the images of image objects OB1 and OB2 from image 303 (i.e., the areas framed by image objects OB1 and OB2). Through the background filling image generated by the aforementioned method, control device 2 generates an image as shown in the example. Figure 3B Image 305.

[0056] It should be understood that, for ease of illustration, images 303 and 305 are only one frame in the image. In actual operation, the control device 2 continuously and dynamically judges one or more frames in the image (e.g., based on 10 frames at the same time). The control device 2 can be adjusted according to the actual operation needs. Those with ordinary knowledge in the art should be able to understand the implementation method for multiple frames based on the above content, so it will not be elaborated further.

[0057] Next, we will describe the speech source focusing function provided by the panoramic imaging conferencing system 1 of the present invention. During a meeting, when a participant speaks, the panoramic imaging conferencing system 1 can focus the image on the speaker (e.g., generate an image focused on the speaker) to improve the quality of the meeting.

[0058] Specifically, the surround-view video conferencing system 1 also includes a microphone array 4, which is connected to a network communication connection to a control device 2 via at least one of a wired connection and a wireless connection. The microphone array 4 is used to sense the sound in the conference room to generate an audio signal. The control device 2 first receives the audio signal from the microphone array 4. Then, based on the audio signal, the control device 2 determines a source of speech (e.g., a conference speaker).

[0059] Subsequently, control device 2 compares the characteristics of multiple meeting participants with the panoramic image corresponding to the source of the speech to determine whether the panoramic image corresponding to the source of the speech contains a first meeting participant characteristic among the multiple meeting participant characteristics. When it is determined that the panoramic image corresponding to the source of the speech contains a first meeting participant characteristic, control device 2 generates a focused image corresponding to the source of the speech. Finally, based on the focused image and the panoramic image, control device 2 generates the panoramic meeting image (e.g., generating a split image, with the upper image being the panoramic image and the lower image being the focused image).

[0060] For example, control device 2 can determine the source of a speech based on direction of arrival (DOA) and beamforming technology. It should be noted that control device 2 determines the source of the speech by comparing whether it matches the characteristics of the multiple meeting participants, thus avoiding misjudgments caused by environmental noise or irrelevant actions and improving the quality of the meeting.

[0061] Furthermore, since conference rooms typically have displays for playing remote video and audio during meetings, to prevent the remote conference audio played on the display from causing the focused image to focus on the display, in some embodiments, the control device 2 limits the scope of the "voice source focusing function" to avoid misjudgment (i.e., the audio emitted by the display 5 must be actively excluded). Specifically, the surround view conferencing system 1 also includes a display 5 for playing remote conference video and remote conference audio. The control device 2 generates an exclusion zone based on a position of the display. The control device 2 compares the voice source with the exclusion zone to determine whether the voice source originates from the exclusion zone. When the voice source originates from the exclusion zone, the focused image corresponding to that voice source is not generated.

[0062] It should be noted that the control device 2 can confirm the position of the display 5 through the aforementioned object identification. In addition, in some embodiments, when the position of the display 5 moves, the control device 2 can also dynamically adjust the exclusion area to avoid misjudgment.

[0063] Next, the loop playback function provided by the panoramic video conferencing system 1 of the present invention will be described. The following paragraphs will describe in detail the implementation details related to the present invention. Before or after the meeting, the panoramic video conferencing system 1 can generate a close-up image of the person participating in the meeting when the image detection system detects the presence of the person participating in the meeting, and start loop playback to introduce the people currently present in the meeting (e.g., loop playback mode).

[0064] Specifically, the panoramic image generating device 3 in the panoramic image conferencing system 1 may also include a first photography device and a second photography device. The first photography device generates a first image, and the second photography device generates a second image. The panoramic image 301 is composed of the first image and the second image.

[0065] In this embodiment, the control device 2 can detect whether a first meeting person with the characteristics of a first meeting person appears in the first image. When the first meeting person with the characteristics of the first meeting person appears in the first image, the control device 2 generates a close-up image of the first meeting person as the first image.

[0066] In addition, the control device 2 detects whether a second meeting person with the characteristics of a second meeting person appears in the second image. When the second meeting person with the characteristics of the second meeting person appears in the second image, the control device 2 generates a close-up image of the second meeting person as the second image.

[0067] Therefore, before or after a meeting, the panoramic video conferencing system 1 can loop the panoramic video 301 composed of the first video and the second video. It should be understood that this invention does not limit the number of photographic devices and videos. Those skilled in the art should be able to understand, based on the above, implementations with different numbers of photographic devices and videos (i.e., more than 2), and therefore will not be elaborated further.

[0068] In some embodiments, the panoramic video conferencing system 1 may further include a cloud server. The panoramic video conferencing system 1 can transmit meeting data (e.g., panoramic video of the current meeting, related information, etc.) to the cloud server via a wired or wireless network, and store the meeting data on the cloud server as historical meeting records for query by relevant personnel.

[0069] As described above, the panoramic video conferencing system 1 provided by this invention analyzes the panoramic image and identifies multiple image objects within it. Furthermore, based on a privacy mode, the panoramic video conferencing system 1 can select one image object to be removed from the panoramic image. Finally, a panoramic meeting image is generated based on the panoramic image and the background-filled image. The panoramic video conferencing system 1, through the aforementioned mechanism, can selectively hide some image objects, providing enterprises with more flexible privacy requirements. In addition, the panoramic video conferencing system 1 offers many other functions, enhancing the service and quality of online meetings.

[0070] The second embodiment of the present invention is a panoramic video conferencing method, the flowchart of which is depicted in... Figure 4The panoramic video conferencing method 400 is applicable to a panoramic video conferencing system, such as the panoramic video conferencing system 1 described in the first embodiment. This panoramic video conferencing system includes a panoramic image generating device and a control device, such as the panoramic image generating device 3 and control device 2 described in the first embodiment. The panoramic video conferencing method 400 is executed by the control device and generates panoramic conferencing images through steps S401 to S407.

[0071] In step S401, the control device analyzes a panoramic image received from the panoramic image generating device to identify multiple image objects in the panoramic image. In step S403, the control device selects an image object to be removed from the multiple image objects based on a privacy mode.

[0072] Next, in step S405, the control device removes an image corresponding to the object to be removed from the panoramic image. Finally, in step S407, the control device generates a panoramic conference image based on the panoramic image and a background filling image.

[0073] In some embodiments, the control device further stores an image recognition model, and the panoramic image conferencing method 400 further includes the following steps: identifying the plurality of image objects in the panoramic image based on an edge detection algorithm and the image recognition model, wherein the image recognition model is generated based on the features of the plurality of objects and the features of the plurality of meeting participants.

[0074] In some embodiments, the surround-view conferencing system further includes a microphone array (e.g., microphone array 4 as described in the first embodiment), connected to the control device via at least one of a wired connection and a wireless connection, for sensing sound in a conference room to generate an audio signal. The surround-view video conferencing method 400 further includes the following steps: receiving the audio signal from the microphone array; determining a speaking source based on the audio signal; comparing multiple meeting participant characteristics with the surround-view image corresponding to the speaking source to determine whether the surround-view image corresponding to the speaking source contains a first meeting participant characteristic among the multiple meeting participant characteristics; when it is determined that the surround-view image corresponding to the speaking source contains the first meeting participant characteristic, generating a focused image corresponding to the speaking source; and generating the surround-view conferencing image based on the focused image and the surround-view image.

[0075] In some embodiments, the surround-view conferencing system further includes a display (e.g., display 5 as described in the first embodiment) for playing a remote conferencing video and a remote conferencing audio. The surround-view video conferencing method 400 further includes the following steps: generating an exclusion zone based on a position of the display; comparing the speech source with the exclusion zone to determine whether the speech source originates from the exclusion zone; and when the speech source originates from the exclusion zone, not generating the focused image corresponding to that speech source.

[0076] In some embodiments, the panoramic image generating apparatus further includes: a first photographing device and a second photographing device, the first photographing device generating a first image, the second photographing device generating a second image, and the panoramic image being composed of the first image and the second image.

[0077] In some embodiments, the panoramic video conferencing method 400 further includes the following steps: detecting whether a first meeting person with the characteristics of a first meeting person appears in the first image; and when the first meeting person with the characteristics of the first meeting person appears in the first image, generating a close-up image of the first meeting person as the first image.

[0078] In some embodiments, the panoramic video conferencing method 400 further includes the following steps: detecting whether a second meeting person with the characteristics of a second meeting person appears in the second image; and when the second meeting person with the characteristics of the second meeting person appears in the second image, generating a close-up image of the second meeting person as the second image.

[0079] In some embodiments, the panoramic image conferencing method 400 further includes the following steps: calculating multiple color feature values ​​of multiple neighboring blocks of the image object to be removed; and generating the background filling image based on the multiple color feature values.

[0080] In addition to the steps described above, the second embodiment can also perform all the operations and steps of the panoramic video conferencing system 1 described in the first embodiment, have the same functions, and achieve the same technical effects. Those skilled in the art to which this invention pertains can directly understand how the second embodiment performs these operations and steps based on the first embodiment described above, has the same functions, and achieves the same technical effects, so it will not be described in detail here.

[0081] It should be noted that in the specification and claims of this invention, certain terms (including: image, meeting participants, characteristics of meeting participants, and photographic device) are preceded by "first" or "second". These "first" and "second" are used only to distinguish different terms. For example, "first" and "second" in "first image" and "second image" are used only to indicate images at different operating times.

[0082] In summary, the panoramic video conferencing technology (including at least a system and method) provided by this invention analyzes the panoramic image and identifies multiple image objects within it. Furthermore, based on a privacy mode, this panoramic video conferencing technology can select an image object to be removed from the panoramic image. Finally, a panoramic meeting image is generated based on the panoramic image and a background-filled image. This panoramic video conferencing technology, through the aforementioned mechanism, allows for the selective hiding of some image objects, providing enterprises with more flexible privacy options. In addition, this panoramic video conferencing technology offers numerous other features, enhancing the service and quality of online meetings.

[0083] The above embodiments are merely illustrative of some implementations of the present invention and to explain the technical features of the present invention, and are not intended to limit the scope and range of protection of the present invention. Any changes or equivalent arrangements that can be easily made by those skilled in the art to which this invention pertains are within the scope of the present invention, and the scope of protection of the present invention is determined by the claims.

Claims

1. A panoramic video conferencing system, characterized in that, Include: A panoramic image generation device for generating panoramic images; A control device is connected to the surround view image generating device and performs the following operations: Analyze the panoramic image to identify multiple image objects within it; Based on a privacy mode, select one image object to be removed from the plurality of image objects; Remove the image corresponding to the object to be removed from the panoramic image; and A panoramic conference image is generated based on the panoramic image and a background filling image; The panoramic video conferencing system described therein also includes: A microphone array, connected to the control device via at least one of a wired connection and a wireless connection, is used to sense the sound in a conference room to generate an audio signal. The control device further performs the following operations: The sound signal is received from the microphone array; Based on the sound signal, the source of the speech can be determined; Compare the characteristics of multiple meeting participants with the surrounding image corresponding to the source of the speech to determine whether the surrounding image corresponding to the source of the speech contains a first meeting participant characteristic among the multiple meeting participant characteristics. When it is determined that the panoramic image corresponding to the source of the speech contains an individual matching the characteristics of the first meeting participant, a focused image corresponding to that source of speech is generated; and The panoramic conference image is generated based on the focused image and the panoramic image; The panoramic video conferencing system described therein also includes: A monitor for displaying video and audio from a remote conference; The control device further performs the following operations: An exclusion zone is generated based on a position of the display; Compare the source of the call with the excluded area to determine whether the call originated from the excluded area; as well as When the source of the speech originates from the excluded area, no focused image is generated corresponding to that source of the speech.

2. The panoramic video conferencing system as described in claim 1, characterized in that, The control device further stores an image recognition model and performs the following operations: Based on an edge detection algorithm and the image recognition model, the multiple image objects in the panoramic image are identified, wherein the image recognition model is generated by training based on the features of multiple objects and the features of multiple meeting participants.

3. The panoramic video conferencing system as described in claim 1, characterized in that, The panoramic image generation device also includes: A first photographic device and a second photographic device, the first photographic device generating a first image and the second photographic device generating a second image, the panoramic image being composed of the first image and the second image.

4. The panoramic video conferencing system as described in claim 3, characterized in that, The control device further performs the following operations: Detect whether a person with the characteristics of a first meeting person appears in the first image; and When a person with the characteristics of the first meeting person appears in the first image, a close-up image of the first meeting person is generated as the first image.

5. The panoramic video conferencing system as described in claim 4, characterized in that, The control device further performs the following operations: Detect whether a second meeting person with the characteristics of a second meeting person appears in the second image; and When a second meeting person with the characteristics of the second meeting person appears in the second image, a close-up image of the second meeting person is generated as the second image.

6. The panoramic video conferencing system as described in claim 1, characterized in that, The control device further performs the following operations: Calculate multiple color feature values ​​of multiple neighboring blocks of the image object to be removed; and The background-filling image is generated based on the multiple color feature values.

7. A panoramic video conferencing method, characterized in that, For a panoramic video conferencing system, the panoramic video conferencing system includes a panoramic video generating device and a control device, the panoramic video conferencing method is executed by the control device and includes the following steps: Analyze a panoramic image received from the panoramic image generating device to identify multiple image objects in the panoramic image; Based on a privacy mode, select one image object to be removed from the plurality of image objects; Remove the image corresponding to the object to be removed from the panoramic image; and A panoramic conference image is generated based on the panoramic image and a background filling image; The panoramic video conferencing system also includes: A microphone array, connected to the control device via at least one of a wired connection and a wireless connection, is used to sense sound in a conference room to generate an audio signal; The panoramic imaging conferencing method also includes the following steps: The sound signal is received from the microphone array; Based on the sound signal, the source of the speech can be determined; Compare the characteristics of multiple meeting participants with the surrounding image corresponding to the source of the speech to determine whether the surrounding image corresponding to the source of the speech contains a first meeting participant characteristic among the multiple meeting participant characteristics. When it is determined that the panoramic image corresponding to the source of the speech contains an individual matching the characteristics of the first meeting participant, a focused image corresponding to that source of speech is generated; and The panoramic conference image is generated based on the focused image and the panoramic image; The panoramic video conferencing system also includes: A monitor for displaying video and audio from a remote conference; The panoramic imaging conferencing method also includes the following steps: An exclusion zone is generated based on a position of the display; Compare the source of the call with the excluded area to determine whether the call originated from the excluded area; as well as When the source of the speech originates from the excluded area, no focused image is generated corresponding to that source of the speech.

8. The panoramic image conferencing method as described in claim 7, characterized in that, The control device further stores an image recognition model, and the panoramic image conferencing method also includes the following steps: Based on an edge detection algorithm and the image recognition model, the multiple image objects in the panoramic image are identified, wherein the image recognition model is generated by training based on the features of multiple objects and the features of multiple meeting participants.

9. The panoramic image conferencing method as described in claim 7, characterized in that, The panoramic image generation device also includes: A first photographic device and a second photographic device, the first photographic device generating a first image and the second photographic device generating a second image, the panoramic image being composed of the first image and the second image.

10. The panoramic image conferencing method as described in claim 9, characterized in that, The panoramic imaging conferencing method also includes the following steps: Detect whether a person with the characteristics of a first meeting person appears in the first image; and When a person with the characteristics of the first meeting person appears in the first image, a close-up image of the first meeting person is generated as the first image.

11. The panoramic image conferencing method as described in claim 10, characterized in that, The panoramic imaging conferencing method also includes the following steps: Detect whether a second meeting person with the characteristics of a second meeting person appears in the second image; and When a second meeting person with the characteristics of the second meeting person appears in the second image, a close-up image of the second meeting person is generated as the second image.

12. The panoramic image conferencing method as described in claim 7, characterized in that, The panoramic imaging conferencing method also includes the following steps: Calculate multiple color feature values ​​of multiple neighboring blocks of the image object to be removed; and The background-filling image is generated based on the multiple color feature values.

Images