Lens scheduling method and device for multi-camera video conference equipment

Abstract

The application provides a lens scheduling method and device for multi-camera video conference equipment, which comprises a panoramic lens group and a close-up lens group; real-time coordinate positions and dynamic state information of a target portrait are acquired, the real-time coordinate positions being detected based on image data collected by the panoramic lens group in the equipment; when the state information indicates that it is dynamic, a pre-stored lens matching area mapping table is inquired based on the real-time coordinate positions; the lens matching area mapping table is constructed based on preset coverage areas of lenses of the close-up lens group and the panoramic lens group in the equipment; a tracking scheduling instruction is automatically generated according to the inquiry result of the lens matching area mapping table; one or more close-up lenses in the close-up lens group are scheduled to track and shoot the target portrait based on the tracking scheduling instruction. The technical scheme of the application avoids the problem of loss of portrait tracking when lenses are switched, shortens the dynamic tracking response delay, and improves the smoothness of portrait tracking.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance technology, and in particular relates to a method and device for scheduling lenses in a multi-camera video conferencing system. Background Technology

[0002] Currently, video conferencing is becoming increasingly widespread, and multi-camera video conferencing equipment has become a core tool in government and enterprise offices, education and training scenarios, and more. Participants need clear and stable images to accurately capture the speaker's expressions and movements, ensuring efficient information transmission. Therefore, the requirements for the smoothness and stability of dynamic human tracking are constantly increasing.

[0003] In existing technologies, multi-camera video conferencing equipment generally suffers from issues of unevenness and instability during human face tracking. In scenarios involving human movement or multiple people taking turns speaking, image shake is prone to occur when switching cameras, and tracking interruptions are also frequent.

[0004] This issue prevents remote participants from clearly obtaining the speaker's status, interferes with information transmission, severely impacts the meeting viewing experience, and makes it difficult to meet the needs of high-quality video conferencing.

[0005] In summary, there is an urgent need for a technical solution that can coordinate the relationship between camera views and optimize dynamic scheduling mechanisms to solve problems such as tracking interruption and image shaking, and improve the dynamic portrait tracking effect. Summary of the Invention

[0006] This application aims to provide a lens scheduling method and device for multi-camera video conferencing equipment to solve the above-mentioned problems, avoid tracking interruption and image jitter, and improve the smoothness of human face tracking.

[0007] To address the aforementioned problems, in a first aspect, the present invention provides a lens scheduling method for a multi-camera video conferencing device. The method is applied to a multi-camera video conferencing device comprising a panoramic lens group and a close-up lens group, the method comprising: The real-time coordinates and status information of the target image are obtained. The real-time coordinates are detected based on the image data collected by the panoramic lens group in the device. When the status information indicates dynamic, the query result is obtained by querying the pre-stored lens matching area mapping table based on the real-time coordinate position; the lens matching area mapping table is constructed based on the preset coverage area of ​​each lens in the close-up lens group and panoramic lens group in the device; Based on the query results of the lens matching area mapping table, actively generate tracking and scheduling instructions; Based on the tracking and scheduling instructions, one or more close-up shots in the close-up shot group are scheduled to track and shoot the target portrait.

[0008] In some embodiments, the process of constructing the lens matching region mapping table includes dividing the coverage partition set based on the preset coverage area of ​​each lens in the close-up lens group and the preset coverage area of ​​each lens in the panoramic lens group. The coverage partition set includes a single lens coverage partition and multiple lens overlapping coverage partitions. Determine the lens set corresponding to each coverage partition in the coverage partition set. The lens set records all lens information that can cover the corresponding partition. Based on the matching and integration of the coverage partition set and the corresponding shot set of each partition, a shot matching region mapping table is constructed. The shot matching region mapping table is used to represent the mapping relationship between the coverage partition and the corresponding shot.

[0009] In some embodiments, when the status information indicates dynamic, querying the pre-stored shot matching region mapping table based on the real-time coordinate position to obtain the query result includes: querying the shot matching region mapping table based on the real-time coordinate position, determining the target coverage partition covering the real-time coordinate position, and obtaining the candidate shot set corresponding to the target coverage partition; under a preset shot tracking scheduling strategy, filtering the candidate shot set to obtain a shot scheduling set, and using the shot scheduling set as the query result.

[0010] In some embodiments, based on tracking scheduling instructions, scheduling one or more close-up shots in a close-up shot group to track and shoot a target portrait includes: when the target portrait is located in an overlapping area covered by at least two close-up shots, controlling all close-up shots covering the overlapping area to simultaneously track and shoot.

[0011] In some embodiments, the method further includes: controlling a first close-up and a second close-up in a close-up group to acquire first image data and second image data respectively; wherein the first close-up and the second close-up have different focal lengths and / or different field of view, and the parameters of the first image data and the second image data are matched with the corresponding lenses.

[0012] In some embodiments, the method further includes: acquiring close-up image data captured by at least one lens in the close-up shot group, and obtaining a target tracking image based on the close-up image data through at least one of translation, cropping, and stitching.

[0013] In some embodiments, the method further includes: acquiring corresponding wide-angle image data based on each lens in the panoramic lens group, determining an overlapping area based on the preset coverage area of ​​each lens, and combining the overlapping area with the wide-angle image data to generate a panoramic image covering the conference room through at least one of translation, cropping, and stitching.

[0014] In some embodiments, the method further includes generating a full-lens scheduling instruction when the state information is static; the full-lens scheduling instruction is used to schedule each lens in the close-up shot group and each lens in the panoramic shot group to simultaneously perform framing and shooting.

[0015] In some embodiments, the step of obtaining the real-time coordinate position and dynamic status information of the target portrait, wherein the real-time coordinate position is detected based on image data collected by the panoramic lens group in the device, includes: obtaining the voice positioning information of the target portrait, and performing portrait detection and tracking based on the voice positioning information and image data of the target portrait to obtain the real-time coordinate position and status information of the target portrait.

[0016] In another aspect, the present invention provides a multi-camera video conferencing device, comprising: a panoramic lens group for acquiring wide-angle image data; A close-up group, comprising at least two close-up shots with different preset coverage areas, is used to acquire close-up image data; The image processing module is used to process wide-angle image data and close-up image data to obtain panoramic images and target tracking images; The human face detection and tracking module is used to detect and track human faces based on panoramic images, and outputs the real-time coordinates and status information of the target human face. The lens scheduling control module has a pre-stored lens matching area mapping table. When the status information is dynamic, it queries the lens matching area mapping table according to the real-time coordinate position and actively generates tracking scheduling instructions according to the preset scheduling strategy to schedule the corresponding close-up shots for tracking and shooting. The lens matching area mapping table is constructed based on the spatial relationship of the preset coverage areas of each lens in the close-up lens group and the panoramic lens group, and is used to map real-time coordinate positions to one or more candidate close-up lenses.

[0017] In some embodiments, the close-up lens group includes a first close-up lens and a second close-up lens. The first close-up lens is located in the middle area of ​​the device body and is used to output a high-definition image of a person at a distance in the middle area of ​​the conference room and to achieve image tracking in that area; when the status information changes, the pitch angle of one or more close-up lenses in the close-up lens group is adjusted so that the head area of ​​the image is kept within the field of view recognition area; the second close-up lens is located in a position parallel to the other close-up lenses and has a different focal length and / or a different field of view than the other close-up lenses.

[0018] In some embodiments, the panoramic lens group includes two panoramic lenses with intersecting optical axes, and the fields of view of the two panoramic lenses have an overlap area of ​​not less than 15° horizontal field of view.

[0019] In some embodiments, the lens scheduling control module is further configured to construct a lens matching region mapping table, specifically including dividing the coverage area into a set of coverage partitions based on the preset coverage areas of each lens in the close-up lens group and the preset coverage areas of each lens in the panoramic lens group, the set of coverage partitions including a single lens coverage partition and multiple lens overlapping coverage partitions; determining the set of lenses that can cover each coverage partition; establishing a mapping relationship between the set of coverage partitions and the set of lenses corresponding to each partition, and completing the construction of the lens matching region mapping table. The generated lens matching region mapping table is used to query the target coverage partition in real time.

[0020] In some embodiments, the image processing module is used to process the wide-angle image data and close-up image data to obtain a panoramic image and a target tracking image, including: the image processing module is used to receive close-up image data acquired by at least one lens in the close-up lens group, and generate a target tracking image by at least one of the processing methods of translation, cropping or stitching; the image processing module is also used to receive wide-angle image data acquired by each lens in the panoramic lens group, and generate a panoramic image covering the conference room by processing the wide-angle image data by at least one of the processing methods of translation, cropping or stitching according to the overlapping area determined by the preset coverage area of ​​each lens.

[0021] In some embodiments, the human face detection and tracking module includes a voice acquisition unit; the voice acquisition unit is used to acquire the voice positioning information of the target human face; the human face detection and tracking module is specifically used to narrow the detection range of the panoramic image based on the voice positioning information, and to perform human face detection and tracking in combination with the panoramic image after the narrowing range, so as to obtain the real-time coordinate position and status information of the target human face.

[0022] This invention first utilizes a wide-angle panoramic lens array to stably acquire blind-spot-free panoramic visual information of the conference room, establishing a complete spatial coordinate reference for subsequent portrait localization. Based on this, the system constructs a lens matching area mapping table according to the preset coverage areas of each lens, thereby pre-associating physical spatial locations with close-up shots that can cover those locations. When the target portrait moves, the system actively queries this mapping table based on its real-time coordinates in the panoramic view, predicting in advance which lens coverage area it will enter and triggering scheduling instructions for the corresponding lens. This proactive query and prediction mechanism based on mapping relationships replaces the passive switching logic relying on image integrity in traditional solutions, allowing scheduling instructions to be issued in advance or simultaneously when the portrait crosses different lens fields of view, achieving continuous handover of tracking tasks. Furthermore, by controlling the collaborative work of multiple lenses in overlapping areas through scheduling strategies, the image jitter caused by refocusing and turning during single lens switching is further reduced, thus achieving a smooth, stable, and uninterrupted portrait tracking experience in wide-area scenarios. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the steps of a camera scheduling method for a multi-camera video conferencing device according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a multi-camera video conferencing device according to an embodiment of this application; Figure 3 A schematic diagram comparing the vertical coverage range of a telephoto lens tilted upwards in a multi-camera video conferencing device, provided as an embodiment of this application; Figure 4 A schematic diagram of a panoramic lens group dynamic tracking scene provided in an embodiment of this application for a multi-camera video conferencing device; Figure 5 A schematic diagram illustrating the layout of a panoramic lens group, a close-up lens group, an integrated conference tablet, and peripheral cameras in a multi-camera video conferencing device according to an embodiment of this application; Figures 6A-6D A schematic diagram illustrating the layout of a panoramic lens group and a close-up lens group in a multi-camera video conferencing device according to an embodiment of this application; Figure 7 A schematic diagram comparing the shooting areas of a medium-telephoto lens of a multi-camera video conferencing device, showing the area far from and near the central main lens, according to an embodiment of this application; Figure 8 A schematic diagram illustrating the spacing between telephoto and wide-angle lenses in a horizontal conference room of a multi-camera video conferencing device, provided as an embodiment of this application; Figure 9 A schematic diagram illustrating the spacing between telephoto and wide-angle lenses in a vertical conference room of a multi-camera video conferencing device, provided as an embodiment of this application; Figure 10 This is a schematic diagram showing the spacing between telephoto and wide-angle lenses in a vertical conference room of a multi-camera video conferencing device, as provided in one embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] With the surge in demand for remote collaboration, video conferencing equipment has become a core tool in government and enterprise offices, education and training, and other scenarios. However, traditional multi-camera video conferencing equipment has revealed two key pain points in practical applications, severely restricting the meeting experience and collaboration efficiency: On the one hand, traditional equipment focuses on static scene coverage, with each camera working independently and generating a viewfinder simply by stitching together images from multiple lenses. This design, which lacks overall coordination of the lens's field of view, is prone to blind spots in wide-area scenarios such as large conference rooms, and cannot respond to human movement in real time. It fails to meet the core requirement of "dynamically tracking the speaker," resulting in untimely and incomplete capture of dynamic images of key figures in meetings.

[0026] On the other hand, while existing improvement solutions attempt to adopt a dual-camera architecture of "panoramic lens + gimbal lens," using the panoramic lens for wide-area monitoring and human detection, and the gimbal lens for close-up tracking, the issue of tracking smoothness remains unresolved. The lens switching logic relies on "image integrity judgment," passively switching to the panoramic lens output only when the gimbal lens cannot fully display the face recognized by the panoramic lens. This passive switching mechanism lacks advance coordination and transition design between lenses, easily leading to interruptions in human tracking and causing image jitter due to parameter adjustment delays during lens switching. This problem is even more pronounced in scenarios where multiple people speak alternately and human images move frequently, severely impacting the meeting viewing experience.

[0027] In-depth analysis reveals that the current technological bottlenecks in multi-camera video conferencing equipment essentially stem from the industry's limited understanding of "lens collaboration logic." Traditional solutions have always focused on the core design concept of a single lens covering a single area, resulting in a technical path where each lens operates independently and switching relies on post-event judgment. This path dependence has led the industry to focus on improving the image quality of individual lenses for a long time, without systematically examining the importance of the correlation between the coverage areas of lenses and the prediction of dynamic switching. It has also ignored the need for a smooth transition when the target image moves between the coverage areas of different lenses, ultimately resulting in insufficient scene coverage integrity and positioning reliability for dynamic portrait tracking.

[0028] Against this backdrop, the inventors realized that the key to solving the problem was not simply improving the performance of individual lenses, but rather reconstructing the lens collaboration and scheduling logic. This involved moving beyond the traditional mindset of passive switching and proactively scheduling and smoothly switching lenses by establishing relationships between lens coverage areas in advance and predicting the movement trajectory of the subject. Based on this understanding, this embodiment provides a multi-camera video conferencing device designed to address issues such as dynamic subject tracking interruptions and image jitter by coordinating lens collaboration and optimizing switching mechanisms. This improves the smoothness of subject tracking and meets the practical needs of high-quality video conferencing.

[0029] Please see Figure 1 This embodiment provides a lens scheduling method for a multi-camera video conferencing device, applicable to a multi-camera video conferencing device including a panoramic lens group and a close-up lens group. The method includes the following steps: Step S1: Obtain the real-time coordinate position and status information of the target image. The real-time coordinate position is detected based on the image data collected by the panoramic lens group in the device. Step S2: When the status information indicates dynamic, query the pre-stored lens matching area mapping table based on the real-time coordinate position to obtain the query result; wherein, the lens matching area mapping table is constructed based on the preset coverage area of ​​each lens of the close-up lens group and the panoramic lens group in the device; Step S3: Based on the query results of the lens matching area mapping table, actively generate tracking and scheduling instructions; Step S4: Based on the tracking scheduling command, schedule one or more close-up shots in the close-up shot group to track and shoot the target portrait.

[0030] In the above embodiments, the panoramic lens group, relying on the advantage of a fixed viewing angle, stably acquires wide-angle image data and generates panoramic images with a large coverage area. This effectively solves the problem of field of view coverage in large conference room scenarios, eliminating blind spots and fully presenting the distribution of people throughout the conference room, providing comprehensive visual data support for subsequent accurate positioning of target people. Simultaneously, by calling the close-up lens group, clear close-up images can be output for target people in different areas, meeting the need for viewing details of people in conference scenarios.

[0031] Furthermore, this embodiment constructs a lens matching area mapping table by pre-setting the field of view coverage area of ​​each lens, clearly establishing the correspondence between lenses and coverage areas, and clarifying the association logic of the field of view coverage of each lens. Based on this, combined with the real-time coordinate position of the target human image, the mapping table can be quickly queried and the corresponding lens matched, automatically generating tracking scheduling instructions. This process pre-plans the switching logic when the target human image moves between different lens coverage areas, avoiding the tracking interruption problem caused by the lack of a multi-camera switching prediction mechanism in existing technologies, while reducing image jitter caused by parameter adjustment delays during lens switching, ultimately improving the smoothness of human image tracking.

[0032] It should be noted that the status information refers to the movement of the target image. If the target image moves, the status signal is dynamic; if the target image does not move, the status signal is static.

[0033] In one embodiment, the process of constructing the lens matching region mapping table includes: dividing the preset coverage areas of each lens in the close-up lens group and the preset coverage areas of each lens in the panoramic lens group into a coverage partition set, the coverage partition set including a single lens coverage partition and multiple lens overlapping coverage partitions; determining the lens set corresponding to each coverage partition in the coverage partition set, the lens set recording all lens information that can cover the corresponding partition; and matching and integrating the coverage partition set and the lens set corresponding to each partition to construct the lens matching region mapping table, which is used to represent the mapping relationship between the coverage partition and the corresponding lens.

[0034] In the above embodiments, by setting a set of coverage partitions including single-lens coverage partitions and multi-lens overlapping coverage partitions, and constructing a lens matching region mapping table by associating the lens sets corresponding to each partition, the lens adaptation logic under different coverage scenarios is clearly defined. Based on this, combined with the target portrait real-time coordinate position query mapping table, the target coverage partition and candidate lens set can be quickly located. Then, the lens scheduling set generation instruction is filtered through a preset scheduling strategy. This process pre-plans the switching path of the portrait moving between the single-lens area and the overlapping area, avoiding the tracking discontinuity problem caused by the lack of clear association between lens coverage partitions in the existing technology. At the same time, the parameter adaptation time during lens switching is reduced by shooting multiple lenses synchronously in the overlapping area, ultimately improving the smoothness of portrait tracking. For example, if the coverage areas of the first close-up and the second close-up overlap, when the portrait enters the overlapping area from the telephoto coverage area, the second close-up has already completed parameter coordination with the first close-up. When the portrait then enters the second close-up's separate coverage area from the overlapping area, seamless shooting can be achieved, avoiding the image stuttering or loss of detail caused by parameter adaptation delay during traditional single-lens switching, further ensuring the smoothness of dynamic tracking.

[0035] In some embodiments, when the status information indicates dynamic, querying a pre-stored lens matching area mapping table based on the real-time coordinate position to obtain the query result includes: querying the lens matching area mapping table based on the real-time coordinate position, determining the target coverage partition covering the real-time coordinate position, and obtaining the candidate lens set corresponding to the target coverage partition. Under the preset shot tracking scheduling strategy, a shot scheduling set is obtained by filtering from the candidate shot set, and the shot scheduling set is used as the query result.

[0036] In the above embodiments, when the target portrait is in a dynamic state, the lens matching area mapping table is first queried based on the real-time coordinate position to determine the target coverage area and the corresponding candidate lens set. Then, a lens scheduling set is selected from the candidate lens set according to the preset lens tracking scheduling strategy to generate a tracking scheduling instruction. This process provides one or more sets of lenses that are suitable through the candidate lens set, avoiding the problem of not being able to achieve smooth switching due to lens parameter mismatch or other portrait capture needs of the lens under a single lens selection. At the same time, the preset scheduling strategy can be combined with scene requirements, such as prioritizing the selection of high-definition lenses or prioritizing the matching of low-latency lenses, to accurately select the optimal lens, reduce the time spent on temporarily judging the lens suitability in dynamic scenes, avoid tracking interruption caused by lens selection delay, further ensure the continuity and image quality of dynamic portrait tracking, and help improve the smoothness of portrait tracking.

[0037] In some embodiments, based on tracking scheduling instructions, one or more close-up shots in a close-up shot group are scheduled to track and shoot the target portrait, including controlling all close-up shots covering the overlapping area to simultaneously track and shoot when the target portrait is located in an overlapping area covered by at least two close-up shots.

[0038] In the above embodiments, by working synchronously with multiple lenses in overlapping areas, the tracking interruption caused by the delay in lens switching judgment that may occur when selecting one lens from multiple lenses in traditional solutions is avoided. Moreover, there is no need to temporarily select a single lens, and the synchronous acquisition of multiple lenses in advance can seamlessly connect the human movement trajectory. On the other hand, the synchronous shooting of multiple lenses provides redundant data support for subsequent image processing. Image details can be optimized through image fusion, reducing image cropping or distortion caused by the limitations of a single lens due to the angle of view and focal length. At the same time, it avoids the parameter adaptation deviation that may occur when "selecting a single lens", further reducing image jitter during lens switching, and ultimately improving the continuity and smoothness of dynamic human tracking.

[0039] In some embodiments, the method further includes: controlling a first close-up and a second close-up in a close-up group to acquire first image data and second image data respectively; wherein the first close-up and the second close-up have different focal lengths and / or different field of view, and the parameters of the first image data and the second image data are matched with the corresponding lenses.

[0040] In the above embodiments, by controlling the first and second close-up shots with different focal lengths and / or field of view in the close-up shot group, first image data and second image data with matching parameters are collected respectively, thus constructing a differentiated image acquisition system. Based on this, it can specifically cover different distances and ranges of scenes in the conference room, avoiding the limitations of a single lens being "blurry at a distance and narrow at close range." It can capture details of distant portraits with lenses with adapted focal lengths, and present the full view of close-up scenes with lenses with adapted field of view, filling the shooting gaps in different scenarios and ultimately improving the completeness of information coverage in the conference screen.

[0041] Furthermore, the above embodiments utilize differentiated designs between the first and second close-up lenses, allowing their coverage areas to complement or partially overlap. Based on this, when the target image moves in different areas, the two types of lenses can each provide image data adapted to the scene, providing redundant data support for lens switching and avoiding tracking stuttering caused by data interruption from a single lens. Simultaneously, the dual data acquisition in overlapping areas provides backup data in case of sudden image quality issues with a single lens, ensuring tracking continuity and ultimately improving the stability of image tracking.

[0042] In some embodiments, the method further includes: acquiring close-up image data captured by at least one lens in the close-up shot group, and obtaining a target tracking image based on the close-up image data through at least one of translation, cropping, and stitching.

[0043] In some embodiments, the method further includes: acquiring corresponding wide-angle image data based on each lens in the panoramic lens group, determining an overlapping area based on the preset coverage area of ​​each lens, and combining the overlapping area with the wide-angle image data to generate a panoramic image covering the conference room through at least one of translation, cropping, and stitching.

[0044] In the above embodiments, the panoramic lens group acquires wide-angle image data through each lens, then determines the overlapping area by combining the preset coverage area of ​​each lens, and finally generates a panoramic image of the conference room through at least one of the following processes: translation, cropping, and stitching. This process can rely on the collaborative coverage of multiple lenses to cover the entire conference room, avoiding blind spots caused by the limited field of view of a single lens, and ensuring that the scene and people in every corner of the conference room can be included in the panoramic view. At the same time, the targeted processing of the overlapping area can eliminate the gaps and color deviations when stitching images from multiple lenses, making the final panoramic image coherent and complete, effectively improving the integrity of the conference room scene coverage and the visual consistency of the panoramic image.

[0045] In the above embodiments, the overlapping area is determined based on the preset coverage area of ​​each lens in the panoramic lens group, providing a clear benchmark for the processing of wide-angle image data. During translation, cropping, and stitching operations, the image area to be adjusted can be accurately located based on this benchmark, reducing invalid cropping or stitching misalignment caused by the lack of a clear reference, and avoiding problems such as local image distortion and edge breaks. At the same time, a clear processing benchmark can also simplify the image integration process, shorten the data processing time, and improve the efficiency of panoramic image generation while ensuring the image quality of the panoramic image.

[0046] In the above embodiments, panoramic images are generated by acquiring wide-angle image data through multiple lenses and processing overlapping areas. This not only leverages the wide-area acquisition capabilities of multiple lenses to cover the vast space of a large conference room, filling the gaps in the field of view that a single lens cannot cover, but also allows for seamless integration of images acquired by different lenses through pixel-level fusion processing of overlapping areas. The resulting panoramic image not only fully presents the overall layout and personnel distribution of the conference room, but also ensures natural image transitions, providing a comprehensive and high-quality visual foundation for subsequent human detection and tracking, and helping to accurately obtain the real-time coordinates of target human figures.

[0047] By acquiring wide-angle image data through multiple lenses and processing overlapping areas to generate panoramic images, this approach leverages the wide-area acquisition capabilities of multiple lenses to cover the vast space of a large conference room, filling the gaps in the field of view that a single lens cannot cover. Furthermore, pixel-level fusion processing of overlapping areas ensures seamless integration of images captured by different lenses. The resulting panoramic image not only fully presents the overall layout and personnel distribution of the conference room but also ensures natural image transitions, providing a comprehensive and high-quality visual foundation for subsequent human detection and tracking, and facilitating the accurate acquisition of the real-time coordinates of target individuals.

[0048] In some embodiments, the method further includes generating a full-lens scheduling instruction when the status information is static. The full-lens scheduling instruction is used to schedule the simultaneous shooting of each lens in the close-up shot group and each lens in the panoramic shot group.

[0049] In the above embodiments, when the status information is identified as static, there is no need to continuously perform computationally intensive operations such as portrait coordinate positioning, lens matching, and parameter adjustment required for dynamic tracking. Each lens only needs to maintain stable acquisition of the current coverage area, which can effectively avoid the waste of computing resources in static scenes, reduce the computing load of the device, and keep each lens in a ready state at all times, laying the foundation for rapid response if the status changes to dynamic in the future.

[0050] Furthermore, for static portraits, a full-lens scheduling command drives the panoramic and close-up lens groups to simultaneously capture images. This allows the panoramic lens group to fully preserve the spatial distribution of the overall conference room scene and the static portrait, while the close-up lens group captures portrait details. Simultaneous acquisition of data from both types of lenses avoids the limitations of relying on single-lens data to simultaneously capture both the overall scene and detailed presentation.

[0051] Meanwhile, in the above embodiments, scheduling all lenses to simultaneously capture images in a static state can avoid parameter adaptation delays caused by the temporary activation of a single lens. Since each lens is always in a continuous acquisition state, when the target portrait changes from static to dynamic, there is no need to wait for lens parameter adjustments. The lens scheduling control module can directly and quickly match the corresponding lens and generate tracking commands based on the acquired image data, which greatly shortens the response time of scene switching and reduces tracking stuttering problems caused by lens activation delays.

[0052] In some embodiments, the step of obtaining the real-time coordinate position and status information of the target portrait, wherein the real-time coordinate position is detected based on image data collected by the panoramic lens group in the device, includes: obtaining the voice positioning information of the target portrait, and performing portrait detection and tracking based on the voice positioning information and image data of the target portrait to obtain the real-time coordinate position and status information of the target portrait.

[0053] Please see Figure 2This embodiment also provides a multi-camera video conferencing device, including a panoramic lens group for acquiring wide-angle image data; A close-up group, comprising at least two close-up shots with different preset coverage areas, is used to acquire close-up image data; The image processing module is used to process wide-angle image data and close-up image data to obtain panoramic images and target tracking images; The human face detection and tracking module is used to detect and track human faces based on panoramic images, and outputs the real-time coordinates and status information of the target human face. The lens scheduling control module has a pre-stored lens matching area mapping table. When the status information is dynamic, it queries the lens matching area mapping table according to the real-time coordinate position and actively generates tracking scheduling instructions according to the preset scheduling strategy to schedule the corresponding close-up shots for tracking and shooting. The lens matching area mapping table is constructed based on the spatial relationship of the preset coverage areas of each lens in the close-up lens group and the panoramic lens group, and is used to map real-time coordinate positions to one or more candidate close-up lenses.

[0054] In the above embodiments, based on the real-time coordinate position, a matching lens covering that real-time coordinate position can be found in the lens matching area mapping table. When actually constructing the lens matching area mapping table, parameters such as the lens's installation position, angle, and focal length need to be comprehensively considered. Complex coordinate transformations and geometric calculations are required to accurately define the coverage area of ​​each lens. When multiple matching lenses exist, it indicates that the real-time coordinate position is located in the overlapping area of ​​the coverage areas of each matching lens. Since the target image is dynamic, it may subsequently leave the overlapping area and enter the coverage area of ​​a matching lens involved in the overlapping area. In this case, to ensure that the target image can continue to be accurately tracked after leaving the overlapping area, this embodiment has already concentrated all matching lenses to simultaneously perform tracking and shooting when the target image enters the overlapping area. An efficient video stream scheduling and allocation mechanism is needed to accurately guide the data of each lens to the corresponding processing unit, avoiding data conflicts and delays. Therefore, the target image can continue to be tracked even after leaving the overlapping area. Thus, this embodiment avoids the loss of image tracking, thereby improving the smoothness of image tracking.

[0055] In some embodiments, the close-up camera group includes a first close-up camera and a second close-up camera. The first close-up camera is positioned in the middle area of ​​the device body and is used to output a high-definition image of a person at a distance in the middle area of ​​the conference room and to achieve person tracking in that area. The middle area of ​​the device body includes the center position of the device body and a certain range around it, rather than just a single center point. When the device's status information changes, it adjusts the pitch angle of one or more close-up cameras in the close-up camera group to keep the head area of ​​the person within the field of view recognition area. The second close-up camera is positioned parallel to the other close-up cameras and has a different focal length and / or a different field of view.

[0056] In the above embodiment, the close-up lens group includes a first close-up lens with a telephoto lens and two second close-up lenses with medium-telephoto lenses. The telephoto lens is positioned in the center of the device body and is used to output a high-definition long-distance portrait of a person in the center area of ​​the conference room and to achieve portrait tracking in the center area. Furthermore, the telephoto lens can be preset to tilt upwards to meet the complete shooting requirements of a standing portrait. Figure 3 The diagram shows a comparison of the vertical coverage range of the telephoto lens tilted upwards. The telephoto lens is deployed at a height of 0.8m, and the angle corresponding to its vertical coverage range is 30°. In this diagram, the telephoto lens tilted upwards at 20° better meets the complete shooting requirements of a standing person than when the lens is not tilted upwards. Two medium-telephoto lenses are set outside the panoramic lens group to output high-definition portraits of people at long distances in the conference room and to achieve medium-to-long-distance portrait tracking. The positions of the two medium-telephoto lenses can be flexibly adjusted according to the conference scenario adapted to the main body of the device.

[0057] In some embodiments, the panoramic lens group includes two lenses arranged at an intersecting angle with the optical axis focusing, and the field of view corresponding to the overlapping area of ​​the field of view captured by the two lenses is not less than 15°.

[0058] In the above embodiments, the panoramic lens group includes two wide-angle lenses. The wider field of view of the wide-angle lenses enables coverage of a large conference room. At the same time, the two wide-angle lenses can quickly fill the gaps in the spatial coverage by simply focusing and cross-arranging them. Combined with an overlap area of ​​no less than 15°, it further eliminates the edge blind spots that are prone to occur with non-wide-angle lenses, ensuring that the entire conference room is presented without any blind spots. Meanwhile, in dynamic portrait tracking scenarios, the wider coverage of the wide-angle lenses makes the path of the target portrait entering the overlapping area of ​​the two lenses smoother and the transition time more sufficient when it moves. This allows for more relaxed simultaneous acquisition of dual lenses and parameter pre-adaptation, significantly improving the smoothness of portrait tracking.

[0059] like Figure 4In real-world dynamic tracking scenarios, the multi-lens collaborative output mechanism in overlapping areas enables seamless switching of the portrait. For example, when a portrait moves from area A (covered only by the wide-angle lens of camera A1) into the overlapping area of ​​areas A and B, this overlapping area is jointly covered by the wide-angle lenses of both camera A1 and camera B1. At this point, the system enters the first stage of collaboration. In this stage, the image output source is synchronously switched to be provided by both camera A1 and camera B1. This design aims to avoid delays and image jitter caused by temporary lens activation or parameter adaptation during subsequent switching by involving camera B1 in the image output process in advance.

[0060] When the human figure is completely within the overlapping area, the system enters the second stage. Cameras A1 and B1 continuously maintain a shared output state, ensuring stable image quality through redundant dual-lens acquisition, thereby avoiding image fragmentation caused by single-lens failure or perspective deviation.

[0061] As the subject begins to leave the coverage area of ​​the wide-angle lens corresponding to camera A1 and gradually enters area B, which is only covered by the wide-angle lens corresponding to camera B1, the system transitions to the third stage. Since camera B1 has already completed parameter matching and image stitching with camera A1 during the collaborative phase in the overlapping area, camera B1 can immediately provide a complete and continuous image after camera A1 exits output, without any additional lens startup or adjustment time. This mechanism ultimately achieves seamless and smooth switching of the subject during cross-lens tracking, completely solving the common problems of image stuttering and tracking interruptions during traditional single-lens switching.

[0062] In some embodiments, a wide-angle lens is positioned in the center of the device body, a telephoto lens is positioned in the center of the device body, and the wide-angle lens and the telephoto lens are arranged side by side vertically.

[0063] In some embodiments, the panoramic lens group and the close-up lens group are positioned below the integrated conference flat panel (TV) with a display, and a camera may also be positioned above the integrated conference flat panel (TV), such as... Figure 5The diagram shows the layout of the panoramic lens group, close-up lens group, integrated conference tablet, and external camera. The diagram includes a panoramic lens group 6, a close-up lens group 7, an integrated conference tablet 8 with a display, and a camera 9 positioned above the integrated conference tablet 8. In this embodiment, when a target person is detected to be within a certain distance of the integrated conference tablet (TV), the image is switched to the camera positioned above. This effectively avoids image truncation caused by close-range obstruction from the panoramic lens group and close-up lens group, without affecting the panoramic image output and tracking functions, and avoids awkward shooting angles for the person. The arrangement of the panoramic lens group and close-up lens group on the integrated conference tablet in Figure 5 is only one embodiment of the present invention; the arrangement of the panoramic lens group and close-up lens group may be referenced elsewhere. Figures 6A-6D .

[0064] In some embodiments, the tilt angle is preset based on the deployment height of the device body.

[0065] In one embodiment, the image acquisition module of the present invention includes a panoramic lens group and a close-up lens group, wherein the panoramic lens group includes a first wide-angle lens 2 and a second wide-angle lens 4, and the close-up lens group includes a telephoto lens 3, a first medium-telephoto lens 1, and a second medium-telephoto lens 5. The first medium-telephoto lens 1, the first wide-angle lens 2, the telephoto lens 3, the second wide-angle lens 4, and the second medium-telephoto lens 5 are arranged on the same horizontal line.

[0066] For example, such as Figure 6A The panoramic lens group and the close-up lens group can be arranged on the same horizontal line along the length of the main body of the equipment, or arranged side by side along the height direction. For example, as shown in Figure 6B, the panoramic lens group (first wide-angle lens 2 and second wide-angle lens 4) is arranged adjacent to each other and is located in the middle area of ​​the device together with the second close-up lens (telephoto lens 3); the second close-up lens (telephoto lens 3) is set to the left or right of the panoramic lens group.

[0067] For example, such as Figure 6C The lenses in the close-up group can be arranged at equal or non-equal intervals to adapt to the shape and seating distribution of different conference rooms. For example, such as Figure 6D The two panoramic lenses in the panoramic lens group can adopt two arc-shaped layouts, which are either concave inward or convex outward, to ensure that the field of view has a horizontal overlap area of ​​not less than 15°. All of the above-mentioned lens layout methods can ensure that the preset coverage area of ​​each lens is reasonably divided, achieve full coverage without blind spots, and provide a stable hardware foundation for the construction of the lens matching area mapping table and dynamic tracking scheduling.

[0068] In the above embodiment, the coverage area of ​​the telephoto lens 3 includes the central and distant area of ​​the conference room, thus enabling high-definition output of distant portraits in the central area of ​​the conference room.

[0069] In the above embodiments, the coverage areas of the first medium telephoto lens 1 and the second medium telephoto lens 5 include areas that can cover long distances in the conference room, thus enabling high-definition output from long distances in the conference room.

[0070] In the exemplary embodiment described above, the first medium telephoto lens 1 and the first wide-angle lens 2 are spaced apart, and the second medium telephoto lens 5 and the second wide-angle lens 4 are spaced apart, so that the first medium telephoto lens 1 and the second medium telephoto lens 5 can cover a wider shooting area through angle adjustment, such as... Figure 7 The diagram shows a comparison of the shooting areas of a medium-telephoto lens far from and close to the main lens. The diagram includes a first medium-telephoto lens 1, a first wide-angle lens 2, a telephoto lens 3, a second wide-angle lens 4, and a second medium-telephoto lens 5. This design is particularly suitable for horizontally oriented conference rooms and horizontally arranged conference tables. This layout ensures coverage of people at medium distances while also covering people in other positions level with the main seat, such as... Figure 8 The diagram shows the spacing between telephoto and wide-angle lenses in a horizontal conference room. The diagram includes a first medium-telephoto lens 1, a first wide-angle lens 2, a telephoto lens 3, a second wide-angle lens 4, and a second medium-telephoto lens 5. If the medium-telephoto lens is placed next to the wide-angle lens, the lens angle required to cover the original area will increase significantly, and angle adjustment alone will not be sufficient to completely cover the original target area.

[0071] In the exemplary embodiment described above, the first medium telephoto lens 1 is positioned horizontally close to the first wide-angle lens 2, and / or the second medium telephoto lens 5 is positioned close to the second wide-angle lens 4. This layout is particularly suitable for long conference rooms arranged longitudinally, such as... Figure 9 The diagram illustrates the spacing between telephoto and wide-angle lenses in a longitudinal conference room. It includes a first medium-telephoto lens 1, a first wide-angle lens 2, a telephoto lens 3, a second wide-angle lens 4, and a second medium-telephoto lens 5. Since seating in a long conference room is primarily distributed along the longitudinal depth, the target area to be covered is relatively concentrated laterally, but exhibits a clear near-far zone longitudinally. In this scenario, placing the medium-telephoto lens close to the wide-angle lens on the same side ensures that their field-of-view axes are essentially aligned laterally, thus providing more concentrated coverage of key areas longitudinally. The medium-telephoto lens can effectively cover personnel at the far longitudinal end of the conference room by adjusting its pitch angle without requiring significant horizontal angle adjustments, simplifying the setup process and improving visual axis stability. Simultaneously, the wide-angle lens covers the overall wide-area environment, including the near longitudinal end.

[0072] In this embodiment, the camera layout is not fixed and can be adaptively configured according to the specific shape of the conference room (wide horizontally or long vertically) and the seating distribution. For horizontally oriented conference rooms, a spaced-out camera layout optimizes horizontal coverage; for vertically oriented conference rooms, a close-to-the-head camera layout optimizes vertical coverage. This flexibility allows this solution to adapt to various complex conference room environments, achieving high-quality acquisition of key areas in all cases.

[0073] In the above embodiment, the coverage area of ​​the first wide-angle lens 2 and the second wide-angle lens 4 shown includes most of the conference room area.

[0074] In some embodiments, the overlapping area between the wide-angle image data acquired by the first wide-angle lens 2 and the second wide-angle lens 4 is used as the redundant area for cropping, and then the panoramic image covering the conference room is obtained through stitching. The field of view of the overlapping area of ​​the coverage area of ​​the first wide-angle lens 2 and the second wide-angle lens 4 is not less than 15°, which can better stitch together a complete human image and improve the reliability of the target human image positioning.

[0075] In the above embodiments, the two lenses are arranged at a cross-focusing angle. The cross-angle is planned based on the lens's field of view characteristics, so that the fields of view captured by the two lenses naturally form an overlapping area in the middle. During image acquisition, the synchronization control unit of the panoramic lens group synchronizes the acquisition frame rate and exposure parameters of the two lenses in real time to avoid brightness and color deviations in the overlapping area image due to parameter differences. When the target portrait enters the overlapping area from the acquisition area of ​​one lens, the main body of the device performs feature point matching and pixel-level fusion processing on the target portrait images output by the two lenses, eliminating edge distortion pixels caused by differences in lens perspective, generating a complete portrait image without stitching traces, and outputting them together. Subsequently, when the target portrait leaves the overlapping area to the acquisition area of ​​another lens, since the other lens is already in the output state, a clear target portrait can be directly and seamlessly output. Furthermore, in the panoramic image generation stage, the image stitching unit uses the overlapping area as a reference to perform edge cropping and smooth stitching on the wide-angle image data of the two lenses, completely eliminating stitching gaps, ensuring that the obtained wide-angle image has no blind spots during stitching, achieving seamless wide-angle coverage, and thus improving the smoothness of portrait tracking.

[0076] In some embodiments, the lens scheduling control module is further configured to construct a lens matching region mapping table, specifically including dividing the coverage area into a set of coverage partitions based on the preset coverage areas of each lens in the close-up lens group and the preset coverage areas of each lens in the panoramic lens group, the set of coverage partitions including a single lens coverage partition and multiple lens overlapping coverage partitions; determining the set of lenses that can cover each coverage partition; establishing a mapping relationship between the set of coverage partitions and the set of lenses corresponding to each partition, and completing the construction of the lens matching region mapping table. The generated lens matching region mapping table is used to query the target coverage partition in real time.

[0077] In some embodiments, the image processing module is used to process the wide-angle image data and close-up image data to obtain a panoramic image and a target tracking image, including: the image processing module is used to receive close-up image data acquired by at least one lens in the close-up lens group, and generate a target tracking image by at least one of the processing methods of translation, cropping or stitching; the image processing module is also used to receive wide-angle image data acquired by each lens in the panoramic lens group, and generate a panoramic image covering the conference room by processing the wide-angle image data by at least one of the processing methods of translation, cropping or stitching according to the overlapping area determined by the preset coverage area of ​​each lens.

[0078] In some embodiments, the human face detection and tracking module includes a voice acquisition unit; the voice acquisition unit is used to acquire the voice positioning information of the target human face; the human face detection and tracking module is specifically used to narrow the detection range of the panoramic image based on the voice positioning information, and to perform human face detection and tracking in combination with the panoramic image after the narrowing range, so as to obtain the real-time coordinate position and status information of the target human face.

[0079] Please see Figure 10 The diagram includes a first medium-telephoto lens 1, a first wide-angle lens 2, a telephoto lens 3, a second wide-angle lens 4, and a second medium-telephoto lens 5. Area A in the diagram is the area covered by the first medium-telephoto lens 1. (x, y) represents the voice location information of the target person. When the voice location information (x, y) is located in area A, the first wide-angle lens 2 identifies the person near the voice location (x, y) and determines the coordinates of that person as the target person's coordinates. The target person's coordinates are reported to the first medium-telephoto lens 1, which then initiates person tracking and detection, tracking the person's traces within the area in real time. When the voice location information (x, y) exceeds the area covered by the first medium-telephoto lens 1, the panoramic lens further determines the area where the target person's coordinates are located. The query results obtained from the lens matching area mapping table guide other lenses to track the target person, thus achieving seamless tracking across areas.

[0080] The above embodiments use the voice positioning information and panoramic image of the target image to perform image detection and tracking, realizing the dual verification and complementarity of audio and vision for image positioning. This greatly improves the anti-interference ability and reliability of image positioning, increases the accuracy of the real-time coordinate position and status information of the target image, and reduces the probability of target tracking loss due to inaccurate image positioning, thereby improving the smoothness of image tracking.

[0081] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for camera scheduling in a multi-camera video conferencing device, characterized in that, The multi-camera video conferencing device includes a panoramic lens group and a close-up lens group, and the method includes: The real-time coordinate position and status information of the target human image are obtained, wherein the real-time coordinate position is detected based on the image data collected by the panoramic lens group in the device; When the status information indicates dynamic, the query result is obtained by querying the pre-stored lens matching area mapping table based on the real-time coordinate position; wherein, the lens matching area mapping table is constructed based on the preset coverage area of ​​each lens of the close-up lens group and the panoramic lens group in the device; Based on the query results of the lens matching region mapping table, a tracking and scheduling instruction is actively generated; Based on the tracking and scheduling instructions, one or more close-up shots from the close-up shot group are scheduled to track and capture the target portrait.

2. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, The process of constructing the lens matching region mapping table includes: Based on the preset coverage areas of each lens in the close-up lens group and the preset coverage areas of each lens in the panoramic lens group, a coverage partition set is obtained, which includes a single lens coverage partition and multiple lens overlapping coverage partitions. Determine the lens set corresponding to each coverage partition in the coverage partition set, wherein the lens set records all lens information that can cover the corresponding partition; Based on the matching and integration of the coverage partition set and the corresponding shot set of each partition, a shot matching region mapping table is constructed. The shot matching region mapping table is used to represent the mapping relationship between the coverage partition and the corresponding shot.

3. The lens scheduling method for a multi-camera video conferencing device according to claim 2, characterized in that, When the status information indicates dynamic, the query result is obtained by querying the pre-stored lens matching area mapping table based on the real-time coordinate position, including: Based on the real-time coordinate location, the lens matching area mapping table is queried to determine the target coverage area covering the real-time coordinate location, and the candidate lens set corresponding to the target coverage area is obtained. Under the preset shot tracking scheduling strategy, a shot scheduling set is obtained by filtering from the candidate shot set, and the shot scheduling set is used as the query result.

4. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, The step of scheduling one or more close-up shots from the close-up group to track and capture the target portrait based on the tracking scheduling command includes: When the target portrait is located in an overlapping area covered by at least two of the close-up shots, control all the close-up shots covering the overlapping area to simultaneously perform tracking shots.

5. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, Also includes: The first and second close-up shots in the close-up shot group are controlled to acquire first image data and second image data respectively; wherein the first close-up shot and the second close-up shot have different focal lengths and / or different field of view, and the parameters of the first image data and the second image data are matched with the corresponding lenses.

6. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, The method further includes: Acquire close-up image data from at least one shot in a close-up shot group, and obtain a target tracking image based on the close-up image data through at least one of translation, cropping, and stitching.

7. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, Also includes: Based on each lens in the panoramic lens group, acquire the corresponding wide-angle image data; Based on the preset coverage area of ​​each lens, the overlapping area is determined, and combined with the overlapping area and the wide-angle image data, a panoramic image covering the conference room is generated through at least one of the following processes: translation, cropping, and stitching.

8. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, The method further includes: When the state information is static, a full-camera scheduling instruction is generated; The full-lens scheduling command is used to schedule all lenses in the close-up lens group and all lenses in the panoramic lens group to simultaneously shoot.

9. The lens scheduling method for a multi-camera video conferencing device according to claim 1, characterized in that, The step of acquiring the real-time coordinate position and status information of the target image, wherein the real-time coordinate position is detected based on image data acquired by the panoramic lens group in the device, includes: The voice positioning information of the target human image is obtained, and the human image detection and tracking are performed based on the voice positioning information and the image data to obtain the real-time coordinate position and status information of the target human image.

10. A multi-camera video conferencing device, characterized in that, include: Panoramic lens group, used to acquire wide-angle image data; A close-up group, comprising at least two close-up shots with different preset coverage areas, is used to acquire close-up image data; The image processing module is used to process the wide-angle image data and close-up image data to obtain panoramic images and target tracking images; The human face detection and tracking module is used to detect and track human faces based on the panoramic image, and output the real-time coordinate position and status information of the target human face. The lens scheduling control module has a pre-stored lens matching area mapping table. When the status information is dynamic, it queries the lens matching area mapping table according to the real-time coordinate position and actively generates a tracking scheduling command according to the preset scheduling strategy to schedule the corresponding close-up shots for tracking and shooting. The lens matching area mapping table is constructed based on the spatial relationship between the preset coverage areas of the close-up lens group and each lens in the panoramic lens group, and is used to map the real-time coordinate position to one or more candidate close-up lenses.

11. A multi-camera video conferencing device according to claim 10, characterized in that, The close-up shot group includes a first close-up shot and a second close-up shot; The first close-up shot is located in the middle area of ​​the main body of the device, and is used to output a high-definition image of a person at a distance in the middle area of ​​the conference room and to realize the person tracking in that area; when the status information changes, the tilt angle of one or more close-up shots in the close-up shot group is adjusted so that the head area of ​​the person remains within the field of view recognition area. The second close-up shot is positioned parallel to the other close-up shots and has a different focal length and / or a different field of view than the other close-up shots.

12. The multi-camera video conferencing device according to claim 10, characterized in that, The panoramic lens group includes two panoramic lenses whose optical axes are intersecting. The two panoramic lenses have overlapping fields of view with a horizontal field of view of not less than 15°.

13. The multi-camera video conferencing device according to claim 10, characterized in that, The shot scheduling control module is also used to construct the shot matching region mapping table, specifically including: Based on the preset coverage areas of each lens in the close-up lens group and the preset coverage areas of each lens in the panoramic lens group, a coverage partition set is obtained, which includes a single lens coverage partition and multiple lens overlapping coverage partitions. Determine the set of lenses that can cover each coverage zone; A mapping relationship is established between the coverage partition set and the lens set corresponding to each partition, and the lens matching region mapping table is constructed. The generated lens matching region mapping table is used to query the target coverage partition in real time.

14. The multi-camera video conferencing device according to claim 10, characterized in that, The image processing module is used to process the wide-angle image data and close-up image data to obtain panoramic images and target tracking images, including: The image processing module is used to receive close-up image data captured by at least one lens in the close-up shot group, and generate a target tracking image through at least one of the processing methods of translation, cropping or splicing. The image processing module is also used to receive wide-angle image data collected by each lens in the panoramic lens group, and to generate a panoramic image covering the conference room by processing the wide-angle image data through at least one of translation, cropping or stitching according to the overlapping area determined by the preset coverage area of ​​each lens.

15. The multi-camera video conferencing device according to claim 10, characterized in that, The human face detection and tracking module includes a voice acquisition unit; The voice acquisition unit is used to acquire the voice positioning information of the target human image; The human face detection and tracking module is specifically used to narrow down the detection range of the panoramic image based on the voice positioning information, and to perform human face detection and tracking in combination with the panoramic image after the narrowing range, so as to obtain the real-time coordinate position and status information of the target human face.

Images