Wireless video conference tracking system
Through the wireless video conference tracking system, the acquisition and adjustment device are used to determine the attitude and sound intensity of the target object, and the acquisition range and direction are adjusted in real time, which solves the low audio efficiency and incomplete picture caused by the speaker walking in the wireless video conference, and improves the Participate in the meeting and experience.
Patent Information
- Application Number
- CN202310627228.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
In wireless video conference, the speaker moves around and the shooting equipment is not fixed and followed in time, resulting in low radio efficiency and incomplete picture transmission, affecting the experience of the participants.
A wireless video conference tracking system is designed to collect image and audio information in the conference space through the acquisition device, determine the posture and sound intensity of the target object, and adjust the image acquisition range and audio acquisition direction according to real-time changes to ensure the image and image of the target object and The completeness and accuracy of audio information.
It improves the attendance experience of participants, ensures the integrity of image information and the accuracy of audio collection, and enhances the acquisition of the main speaker's pictures and voice by remote participants.
Smart Images

Figure CN119071420A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of data acquisition, and in particular to a wireless video conference tracking system. Background Art
[0002] With the development of network technology, users are using wireless video conferencing scenarios more and more frequently to improve work efficiency. However, during a wireless video conference, the speaker may move around a lot, and the camera equipment may not be able to follow the speaker in time, resulting in reduced sound reception efficiency and incomplete image transmission of the speaker, which in turn worsens the conference experience for participants. Summary of the invention
[0003] In view of this, an embodiment of the present invention provides a wireless video conference tracking system to solve the problem that during a wireless video conference, the speaker may move around a lot or the camera equipment may not be able to follow the camera in time, which may lead to low sound reception efficiency and incomplete picture transmission in the wireless video conference, resulting in a poor experience for the participants.
[0004] According to a first aspect, an embodiment of the present invention provides a wireless video conference tracking system, including:
[0005] A collection device, used for collecting image information and audio information in the conference space;
[0006] A first determining device is used to determine, according to the sound intensity in the audio information, a target df object in the image information corresponding to the sound intensity;
[0007] A second determining device, used to determine the spatial posture information of the target object in the conference space;
[0008] The adjusting device is used to adjust the image acquisition range and the audio information acquisition direction for acquiring the image information in the conference space according to the real-time changes of the spatial posture information.
[0009] Optionally, the collection device further includes:
[0010] At least one image acquisition module, at least one sound pickup module and a storage module are arranged in the conference space;
[0011] The image acquisition module is used to acquire image information in the conference space and send the image information in the conference space to the storage module;
[0012] The sound pickup module is used to collect audio information in the conference space and send the audio information in the conference space to the storage module;
[0013] A storage module is used to store the acquired image information and the acquired audio information.
[0014] Optionally, the first determining device includes:
[0015] An object detection module, used to detect the presence of a single acquisition object in the image information in the conference space;
[0016] An audio extraction module, used to extract the intensity of a single sound present in the audio information;
[0017] A mapping module, used to map the single acquisition object with a single sound intensity to obtain a target mapping group;
[0018] The first determination module is used to determine the collection object corresponding to the sound intensity in the target mapping group as the target object.
[0019] Optionally, the first determining device includes:
[0020] An object detection module, used to detect at least one acquisition object in the image information in the conference space;
[0021] An audio extraction module, used to extract at least one sound intensity present in the audio information;
[0022] A mapping module, used to perform spatiotemporal mapping between the at least one sound intensity and the location of the at least one acquisition object, and determine at least one target mapping group corresponding to at least one group of the acquisition objects and the sound intensity;
[0023] a sorting module, configured to sort the at least one target mapping group according to the sound intensity to obtain a sound intensity sorting table, and extract the first sound intensity from the sound intensity sorting table;
[0024] A first determining module is configured to determine a collection object corresponding to the first sound intensity as a target object.
[0025] Optionally, the target detection module includes: a preprocessing submodule, which is used to highlight the edge information of the captured object in the image information in the conference space.
[0026] Optionally, the first determining device further includes: a noise reduction module, configured to filter environmental noise.
[0027] Optionally, the second determining device includes: a posture detection module, configured to perform posture detection on the target object to determine spatial posture information of the target object in the conference space.
[0028] Optionally, the second determining device further includes:
[0029] A posture detection module, used to perform posture detection on the target object to determine current posture information of the target object;
[0030] A comparison module is used to obtain the current posture information of the target object, match the current posture information of the target object with a preset posture in a preset posture set, and when the current posture information successfully matches the preset posture in the preset posture set, determine the current posture information as the spatial posture information of the target object in the conference space.
[0031] Optionally, the adjusting device comprises:
[0032] An acquisition module, used to acquire the spatial posture information at the previous moment and the spatial posture information at the current moment;
[0033] A comparison module is used to compare the spatial posture information at the previous moment and the spatial posture information at the current moment to see whether there is a real-time change in the spatial posture information; if there is a real-time change in the spatial posture information between the spatial posture information at the previous moment and the spatial posture information at the current moment, it is necessary to adjust the image acquisition range and audio information acquisition direction for obtaining the image information in the conference space.
[0034] Optionally, the adjusting device comprises:
[0035] An image adjustment module, used to make the target object be located at the center of the image in real time according to the real-time change of the spatial posture information;
[0036] The audio adjustment module is used to selectively turn on or off the sound pickup module according to the real-time changes of the spatial posture information.
[0037] The wireless video conference tracking system provided by the present application collects image information and audio information in the conference space by setting up a collection device, and then the first determination device and the second device determine the spatial posture information and sound intensity information, and then the adjustment device adjusts the spatial posture information and sound intensity information to ensure that the collection device can collect the image and audio information of the target object in real time, further ensure the integrity of the image information of the target object and improve the accuracy of audio collection, and further improve the conference experience of the participants. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:
[0039] Figure 1 A schematic diagram of the structure of the wireless video conference tracking system provided for the implementation of this application;
[0040] Figure 2 A schematic diagram of the structure of an optional wireless video conference tracking system provided for the implementation of this application;
[0041] Figure 3 A schematic diagram of the structure of a first determining device in a wireless video conference tracking system provided for the implementation of the present application;
[0042] Figure 4 A schematic diagram of the structure of the second determining device in the wireless video conference tracking system provided for the implementation of the present application;
[0043] Figure 5 A schematic diagram of the structure of the first adjustment determination device in the wireless video conference tracking system provided for the implementation of this application;
[0044] Figure 6 A schematic structural diagram of yet another optional wireless video conference tracking system provided for the implementation of the present application. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0046] The present application provides a wireless video conference tracking system, the application scenario of which can be information communication between a single conference space and multiple wireless terminals, or information communication between multiple conference spaces using a wireless network. For ease of understanding, the present application embodiment will take the wireless video conference tracking between a single conference space and multiple wireless terminals as an example, and explain it in combination with the wireless video conference tracking system.
[0047] The wireless video conference tracking conference system provided in the present application is used to track the posture information and voice information of the conference speaker (target object), ensuring that remote participants (participants participating in the conference through wireless terminals) can always obtain the main presentation screen of the conference speaker and the speaker's voice information, so that the remote participants can have a better conference experience. At the same time, by collecting image information and audio information, the integrity of the image information is further guaranteed and the accuracy of the collected audio information is improved.
[0048] This application provides a wireless video conference tracking system, such as Figure 1As shown, the wireless video conference tracking system includes: a collection device 10, a first determination device 11, a second determination device 12 and an adjustment device 13. The collection device 10, the first determination device 11, the second determination device 12 and the adjustment device 13 are connected to each other in a communication manner, and the communication method can be wireless, WiFi, or Bluetooth.
[0049] The collection device 10 is used to collect image information and audio information in the conference space.
[0050] In this embodiment, a plurality of collection devices 10 are arranged in the conference space, and the positions of each collection device 10 do not overlap.
[0051] The first determining means 11 is used to determine the target object in the image information corresponding to the sound intensity according to the sound intensity in the audio information.
[0052] The second determining means 12 is used to determine the spatial posture information of the target object in the conference space.
[0053] The adjusting device 13 is used to adjust the image acquisition range and the audio information acquisition direction for acquiring the image information in the conference space according to the real-time changes of the spatial posture information.
[0054] In this embodiment, the first determination device 11, the second determination device 12 and the adjustment device 13 can be processors for processing different information. For example, the processor corresponding to the first determination device 11 is used to process audio information and judge the sound intensity to determine the target object in the image information corresponding to the sound intensity.
[0055] In this embodiment, the sound intensity in the audio information includes the sound pressure information of the target object; it also includes the sound pressure information other than the target object, so the sound pressure information needs to be sorted to determine the target object.
[0056] The wireless video conference tracking system provided in the present application collects the image information and audio information in the conference space by setting up a collection device 10, and then the first determination device 11 and the second device determine the spatial posture information and the sound intensity information, and then the adjustment device 13 adjusts the spatial posture information and the sound intensity information to ensure the integrity of the image information collected by the collection device 10 and improve the accuracy of the audio collection, thereby improving the experience of the participants.
[0057] Optionally, the first determining device 11 further includes: a noise reduction module, configured to filter environmental noise.
[0058] In this embodiment, since there is noise in the conference space, a noise reduction module needs to be provided to filter out sounds other than human voices, so as to prevent the remote participants from being affected by the noise and having their experience reduced.
[0059] This application provides a wireless video conference tracking system, such as Figure 2 As shown, the collection device 10 in the wireless video conference tracking system includes:
[0060] At least one image acquisition module 101 , at least one sound pickup module 102 and a storage module 103 are arranged in the conference space.
[0061] The image acquisition module 101 is used to acquire image information in the conference space and send the image information in the conference space to the storage module 103 .
[0062] The sound collection module 102 is used to collect audio information in the conference space and send the audio information in the conference space to the storage module 103 .
[0063] The storage module 103 is used to store the acquired image information and audio information.
[0064] In this embodiment, the image acquisition module 101 can be a camera, and the sound pickup module 102 can be a microphone. The image acquisition module 101 and the sound pickup module 102 are set in different directions of the conference space, for example: the five directions of the conference space, namely, east, west, north, south and center. When a target object initiates a meeting in the conference space, multiple wireless terminal devices access the meeting. The image acquisition module 101 closest to the target object collects the target object; the sound pickup module 102 closest to the target object collects the audio of the target object, and then the image acquisition module 101 and the sound pickup module 102 send the collected image information and audio information to the storage module 103 for storage. When the corresponding image information or audio information needs to be extracted, it can be extracted from the memory.
[0065] The wireless video conference tracking system provided in the present application collects the conference information of the conference space by setting up an image acquisition module 101 and a sound pickup module 102, and then sends the conference information to a storage for storage, which can be retrieved when needed, thereby ensuring the flexibility of data extraction.
[0066] This application provides a wireless video conference tracking system, such as Figure 3 As shown, the first determining device 11 in the wireless video conference tracking system includes:
[0067] The object detection module 111 is used to detect the existence of a single acquisition object in the image information in the conference space.
[0068] In this embodiment, a controller equipped with a target detection algorithm can be used to collect image information in the conference space and perform target detection. Preferably, the target detection algorithm can be a Fast R-CNN target detection algorithm, which is based on the idea of region extraction and realizes target detection by first selecting some candidate boxes and then classifying and regressing these candidate boxes. It can detect a single collection object or multiple collection objects.
[0069] The audio extraction module 112 is used to extract the intensity of a single sound in the audio information.
[0070] The mapping module 113 is used to map a single collection object with a single sound intensity to obtain a target mapping group.
[0071] The first determining module 114 is configured to determine the collection object corresponding to the sound intensity in the target mapping group as the target object.
[0072] In this embodiment, when there is only one target object in the conference space, the target detection algorithm in the target detection module 111 can be used to perform target detection on the image information in the conference space to determine the collection object existing in the conference space. Secondly, the audio extraction module 112 is used to extract the sound intensity in the audio information. After that, mapping is performed after the determined single collection object and sound intensity. The audio information is determined as the sound information emitted by the collection object. Finally, the collection object that has completed the mapping is determined as the target object. In this embodiment, the target object can be understood as the speaker of the meeting.
[0073] Optionally, the first determining device 11 may further include:
[0074] The object detection module 111 is used to detect at least one acquisition object in the image information in the conference space.
[0075] The audio extraction module 112 is used to extract at least one sound intensity in the audio information.
[0076] The mapping module 113 is used to perform spatiotemporal mapping between at least one sound intensity and the location of at least one acquisition object, and determine at least one target mapping group corresponding to at least one group of acquisition objects and sound intensities.
[0077] The sorting module is used to sort at least one target mapping group according to the sound intensity to obtain a sound intensity sorting table, and extract the first sound intensity from the sound intensity sorting table.
[0078] The first determining module 114 is configured to determine the collection object corresponding to the first sound intensity as a target object.
[0079] In this embodiment, since the target detection module 111 detects multiple collection objects in the image information, in order to further determine the target object, it is also necessary to correspond the collection object with the sound intensity. Specifically, the specific position of the collection object can be determined through the image information, and the specific position of the microphone can be inferred using the image information. The sound intensity corresponding to the collection object is determined according to the specific position of the collection object, the specific position of the microphone and the sound intensity. Then, the sound intensities corresponding to the collection are sorted, and the collection object corresponding to the first sound intensity is determined as the target object.
[0080] Optionally, the object detection module 111 includes: a preprocessing submodule, which is used to highlight edge information of the captured object in the image information in the conference space.
[0081] In order to quickly extract the collection object, the wireless video conference tracking system provided by the present application needs to use a preprocessing submodule to preprocess the image information in the conference space before using the target detection module 111, wherein the preprocessing operation can be to improve the contrast, brightness and clarity in the image information so that the collection object can be quickly extracted from the conference space later.
[0082] This application provides a wireless video conference tracking system, such as Figure 4 As shown, the second determining device 12 in the wireless video conference tracking system includes:
[0083] The posture detection module 121 is used to perform posture detection on the target object to determine the spatial posture information of the target object in the conference space.
[0084] In this embodiment, the posture detection algorithm in the posture detection module 121 can be used to perform posture detection on the target object, and it has been determined that the target object is in the conference screen. When the posture detection module 121 cannot detect the posture of the target object, it means that the target object has moved out of the conference screen, and the image acquisition device needs to re-determine the target object; when the posture detection module 121 detects the posture of the target object, it means that the target object has not moved out of the conference screen, and the image acquisition device continues to collect images of the target object. Preferably, the posture detection algorithm can be an OpenPose human posture detection algorithm, which uses a convolutional neural network (CNN) to detect human key points, and uses residual connections and feature pyramids in the network to improve detection accuracy. Specifically, the OpenPose human posture detection algorithm sends the input image to different levels of networks for processing to extract feature information of different scales. Then use these feature information to predict the position coordinates of the key points of the human body at each pixel point, and finally obtain the global key point distribution; after the key point detection is completed, the OpenPose human posture detection algorithm will also match and estimate the key points. It matches the detected key points through the previously trained model and deep learning technology, that is, divides these key points into a human body according to certain rules, and estimates the posture of each human body. In the process of posture estimation, the OpenPose human posture detection algorithm will use the previously trained model to infer the posture information of the human body based on the relationship between each key point; finally, the OpenPose human posture detection algorithm converts the key points and posture information into a human skeleton model and outputs it to the conference screen, thereby realizing the posture detection of the target object.
[0085] Optionally, the second determining means 12 may further include:
[0086] The posture detection module 121 is used to perform posture detection on the target object to determine the current posture information of the target object;
[0087] The comparison module 122 is used to obtain the current posture information of the target object, match the current posture information of the target object with the preset posture in the preset posture set, and when the current posture information successfully matches the preset posture in the preset posture set, determine the current posture information as the spatial posture information of the target object in the conference space.
[0088] This application provides a wireless video conference tracking system, such as Figure 5 As shown, the adjustment device 13 in the wireless video conference tracking system includes:
[0089] The acquisition module 131 is used to acquire the spatial posture information of the previous moment and the spatial posture information of the current moment.
[0090] The comparison module 132 is used to compare whether there is a real-time change in the spatial posture information between the previous moment and the current moment; if there is a real-time change in the spatial posture information between the previous moment and the current moment, it is necessary to adjust the image acquisition range and audio information acquisition direction for obtaining the image information in the conference space.
[0091] The image adjustment module 133 is used to make the target object be located at the center of the image in real time according to the real-time changes of the spatial posture information.
[0092] The audio adjustment module 134 is used to selectively turn on or off the sound pickup module 102 according to the real-time changes of the spatial posture information.
[0093] In this embodiment, the spatial posture information of the previous moment and the spatial posture information of the current moment are acquired by the acquisition module 131, and the spatial posture information of the previous moment and the spatial posture information of the current moment are sent to the comparison module 132 for comparison. When the comparison module 132 determines that the spatial posture information has changed, it means that the target object has moved. At this time, it is necessary to capture the target object through the image adjustment module 133 and the audio adjustment module 134 to ensure that the target object is in the center of the conference in real time and the audio intensity is kept at the maximum state in real time.
[0094] The wireless video conference tracking system provided by the present application collects the image information and audio information in the conference space by setting up a collection device 10, and then the first determination device 11 and the second device determine the spatial posture information and the sound intensity information, and then the adjustment device 13 adjusts the spatial posture information and the sound intensity information to ensure that the collection device 10 can collect the image and audio information of the target object in real time, further ensure the integrity of the image information of the target object and improve the accuracy of audio collection, and further improve the conference experience of the participants.
[0095] The embodiment of the present invention also provides a wireless video conference tracking system. Figure 6 As shown, the wireless video conference tracking system may include a processor 61 and a memory 62, wherein the processor 61 and the memory 62 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.
[0096] The processor 61 may be a central processing unit (CPU). The processor 61 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.
[0097] The memory 62 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the modules provided by the wireless video conference tracking system in the embodiment of the present invention. The processor 61 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 62.
[0098] The memory 62 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 61, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 62 may optionally include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to the processor 61 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0099] For details of the wireless video conference tracking system, please refer to Figures 1 to 5 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.
[0100] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A wireless video conference tracking system, characterized in that: include: A collection device, used for collecting image information and audio information in the conference space; A first determining device, configured to determine, according to the sound intensity in the audio information, a target object in the image information corresponding to the sound intensity; A second determining device, used to determine the spatial posture information of the target object in the conference space; The adjusting device is used to adjust the image acquisition range and the audio information acquisition direction for acquiring the image information in the conference space according to the real-time changes of the spatial posture information.
2. The wireless video conference tracking system according to claim 1, characterized in that: The collection device also includes: At least one image acquisition module, at least one sound pickup module and a storage module are arranged in the conference space; The image acquisition module is used to acquire image information in the conference space and send the image information in the conference space to the storage module; The sound pickup module is used to collect audio information in the conference space and send the audio information in the conference space to the storage module; A storage module is used to store the acquired image information and the acquired audio information.
3. The wireless video conference tracking system according to claim 1, characterized in that: The first determining device comprises: An object detection module, used to detect the presence of a single acquisition object in the image information in the conference space; An audio extraction module, used to extract the intensity of a single sound present in the audio information; A mapping module, used to map the single acquisition object with a single sound intensity to obtain a target mapping group; The first determination module is used to determine the collection object corresponding to the sound intensity in the target mapping group as the target object.
4. The wireless video conference tracking system according to claim 1, characterized in that: The first determining device comprises: An object detection module, used to detect at least one acquisition object in the image information in the conference space; An audio extraction module, used to extract at least one sound intensity present in the audio information; A mapping module, used to perform spatiotemporal mapping between the at least one sound intensity and the location of the at least one acquisition object, and determine at least one target mapping group corresponding to at least one group of the acquisition objects and the sound intensity; A sorting module, configured to sort the at least one target mapping group according to the sound intensity to obtain a sound intensity sorting table, and extract the first sound intensity from the sound intensity sorting table; A first determining module is configured to determine a collection object corresponding to the first sound intensity as a target object.
5. The wireless video conference tracking system according to claim 3 or 4, characterized in that: The target detection module comprises: The preprocessing submodule is used to highlight the edge information of the captured object in the image information in the conference space.
6. The wireless video conference tracking system according to claim 1, characterized in that: The first determining device further includes: a noise reduction module, configured to filter environmental noise.
7. The wireless video conference tracking system according to claim 1, characterized in that: The second determining device comprises: The posture detection module is used to perform posture detection on the target object to determine the spatial posture information of the target object in the conference space.
8. The wireless video conference tracking system according to claim 1, characterized in that: The second determining device further includes: A posture detection module, used to perform posture detection on the target object to determine current posture information of the target object; A comparison module is used to obtain the current posture information of the target object, match the current posture information of the target object with a preset posture in a preset posture set, and when the current posture information successfully matches the preset posture in the preset posture set, determine the current posture information as the spatial posture information of the target object in the conference space.
9. The wireless video conference tracking system according to claim 1, characterized in that: The adjusting device comprises: An acquisition module, used to acquire the spatial posture information at the previous moment and the spatial posture information at the current moment; A comparison module is used to compare the spatial posture information at the previous moment and the spatial posture information at the current moment to see whether there is a real-time change in the spatial posture information; if there is a real-time change in the spatial posture information between the spatial posture information at the previous moment and the spatial posture information at the current moment, it is necessary to adjust the image acquisition range and audio information acquisition direction for obtaining the image information in the conference space.
10. The wireless video conference tracking system according to claim 2, characterized in that: The adjusting device comprises: An image adjustment module, used to make the target object be located at the center of the image in real time according to the real-time change of the spatial posture information; The audio adjustment module is used to selectively turn on or off the sound pickup module according to the real-time changes of the spatial posture information.
