Method and apparatus for capturing close-up picture, and display device and storage medium

Abstract

Provided in the present application are a method and apparatus for capturing a close-up picture, and a display device and a storage medium. The method is applied to a remote conference system, which comprises at least one PTZ camera and a display device, wherein the display device is provided with a camera. The method comprises: acquiring an imaging picture collected by a camera; detecting a target area where a close-up object to be photographed is located in the imaging picture; on the basis of the target area and a set mapping relationship, determining a target parameter value of an angle control parameter of a PTZ camera, wherein the mapping relationship comprises associations between areas in the imaging picture of the camera and parameter values of the angle control parameter of the PTZ camera; and on the basis of the target parameter value of the angle control parameter, enabling the PTZ camera to move towards said close-up object, so as to capture a close-up picture of said close-up object. The method can flexibly realize the linkage between a PTZ camera and a camera in a display device, thereby achieving good imaging effects for close-up pictures.

Description

A method, apparatus, display device, and storage medium for capturing close-up images. Technical Field

[0001] This application relates to the field of video processing, and more specifically, to a method, apparatus, display device, and storage medium for capturing close-up images. Background Technology

[0002] In current conferencing scenarios, remote audio and video functions are primarily implemented using the built-in cameras of interactive tablets, such as conference handwriting displays. However, due to the inherent hardware limitations of these cameras (such as lens aperture, imaging distortion, and sensor noise), video conferencing functions suffer from several limitations, including low image resolution, blurry images, and limited imaging distance. To achieve close-up functionality, a common approach is to crop the area of ​​interest from the image obtained by the conference handwriting display's built-in camera, or to use an additional digital zoom lens module to image the area of ​​interest. While these methods can improve the image quality of close-up shots to some extent, the overall image quality remains poor.

[0003] Summary of the Invention

[0004] This application provides a method, apparatus, display device, and storage medium for capturing close-up images. The method can flexibly realize the linkage between the PTZ camera and the camera in the display device, resulting in good imaging effect of the close-up images.

[0005] Firstly, a method for capturing close-up images is provided, applicable to a remote conferencing system. The remote conferencing system includes at least one PTZ camera and a display device, the display device being equipped with a camera. The method includes: acquiring an image captured by the camera; detecting a target area in the image where the object to be captured is located; and determining a target parameter value for the angle control parameters of the PTZ camera based on the target area and a set mapping relationship. The mapping relationship includes the connection between each area in the image captured by the camera and the parameter value of the angle control parameters of the PTZ camera.

[0006] Based on the target parameter value of the aforementioned angle control parameters, the PTZ camera moves toward the object to be zoomed in, and captures a close-up image of the object.

[0007] In the above technical solution, the display device is equipped with at least one PTZ camera. As a device with optical zoom capability, the PTZ camera has the advantages of both a global wide viewing angle and a local close-up staring view. By combining the excellent imaging performance of the PTZ camera with the display device, the linkage between the PTZ camera and the camera built into the display device can be flexibly realized. This can significantly improve the close-up imaging effect in conference scenarios, and there is no need to make the camera built into the display device have a larger numerical aperture, so it will not affect the appearance or industrial design of the display device itself, and it can also be applied to existing display device models. The PTZ camera attitude adjustment method involved in this application differs from both the traditional method of manually and physically adjusting the imaging angle of the PTZ camera and the method of using real-time online calculation of feature matching points to obtain pose adjustment parameters to drive the PTZ camera attitude adjustment. In order to achieve flexible cooperation between the PTZ camera and the display device for imaging conference scenes, this application pre-captures the conference scene using the camera of the display device and divides it into multiple local image regions. The relationship between each local image region and the angle control parameters of the PTZ camera is pre-calculated and stored. This allows the PTZ camera to automatically cooperate with the display device to perform close-up imaging of any conference participant or local scene during real-time video conferencing, ensuring both the presentation quality and efficiency of close-up images.

[0008] In conjunction with the first aspect, in some possible implementations, before detecting the target area where the object to be zoomed in is located in the aforementioned imaging image, the method further includes: receiving a close-up image presentation requirement for the object to be zoomed in.

[0009] In conjunction with the first aspect, in some possible implementations, the aforementioned remote conferencing system further includes: at least one other display device that is communicatively connected to the aforementioned display device, and the aforementioned method further includes: obtaining a close-up image captured by the aforementioned PTZ camera and sending the close-up image to the aforementioned at least one other display device for display.

[0010] In conjunction with the first aspect, in some possible implementations, the aforementioned mapping relationship includes: the connection between the pixel coordinates of each region in the image frame of the aforementioned camera and the parameter values ​​of the angle control parameters of the aforementioned PTZ camera; the determination of the target parameter value of the angle control parameters of the aforementioned PTZ camera based on the aforementioned target region and the set mapping relationship includes: determining the target pixel coordinates of the aforementioned target region in the image frame of the aforementioned camera; and mapping the aforementioned target pixel coordinates into the aforementioned mapping relationship to obtain the target parameter value of the angle control parameters of the aforementioned PTZ camera.

[0011] In the aforementioned technical solution, by establishing a relationship between the pixel coordinates of each region in the camera's image and the angle control parameters of the PTZ camera, it is convenient to directly calculate the target parameter value by substituting the target pixel coordinates into the mapping relationship. This allows the PTZ camera to output close-up images of the region represented by the target pixel coordinates. Furthermore, the angle control parameters of the PTZ camera are essentially mechanical motion control parameters. By mapping intuitive pixel coordinates to complex mechanical motion control parameters, the intelligence level of the remote conferencing system is greatly improved.

[0012] In combination with the first aspect and the above implementation methods, in some possible implementation methods, the angle control parameters include: pitch angle and horizontal rotation angle; the mapping relationship includes a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; the method of substituting the target pixel coordinates into the mapping relationship to obtain the parameter value of the angle control parameters of the PTZ camera includes: substituting the target pixel coordinates into the first mapping relationship to obtain the target parameter value of the pitch angle of the PTZ camera; and substituting the target pixel coordinates into the second mapping relationship to obtain the target parameter value of the horizontal rotation angle of the PTZ camera.

[0013] In the above technical solution, corresponding first and second mapping relationships are set for the pitch angle and horizontal rotation angle of the PTZ camera, respectively. This facilitates the rapid and accurate acquisition of the target parameter value of the pitch angle through the first mapping relationship, and the rapid and accurate acquisition of the target parameter value of the horizontal rotation angle through the second mapping relationship. Furthermore, based on these target parameter values ​​for the pitch and horizontal rotation angles, the PTZ camera is controlled to move towards the object to be zoomed in on. This allows for precise control of the PTZ camera in both horizontal and vertical dimensions, enabling the PTZ camera to quickly move to the position aligned with the object, thus improving the efficiency and accuracy of outputting close-up images of the object.

[0014] Combining the first aspect and the above implementation methods, in some possible implementation methods, the above mapping relationship is determined in the following way: constructing a mapping expression for the above mapping relationship; wherein the above mapping expression includes parameters to be calibrated; acquiring multiple sets of sampled data; wherein each set of the above sampled data includes: when the above PTZ camera is facing a direction, the sampled value of the angle of the above PTZ camera and the sampled value of the pixel coordinates corresponding to the above camera, wherein the sampled value of the pixel coordinates refers to: when the above PTZ camera is facing the above direction, the pixel coordinates of a specified pixel in the image of the above PTZ camera are mapped to the mapped coordinates in the image of the above camera; based on the above multiple sets of sampled data and the above mapping expression, solving for the calibration values ​​of the parameters to be calibrated, and obtaining the above mapping relationship.

[0015] In conjunction with the first aspect and the above implementation methods, in some possible implementation methods, the acquisition of multiple sets of sampling data includes: acquiring the image of the PTZ camera and the image of the camera when the PTZ camera is facing a certain direction, and performing feature point matching on the image of the PTZ camera and the image of the camera to calculate the homography transformation matrix between the PTZ camera and the camera when the PTZ camera is facing the aforementioned direction; mapping the pixel coordinates of a specified pixel in the image of the PTZ camera to the image of the camera based on the homography transformation matrix when the PTZ camera is facing the aforementioned direction, obtaining the mapped coordinates, and using the mapped coordinates as the sampled values ​​of the pixel coordinates corresponding to the camera; using the angle value of the PTZ camera when it is facing the aforementioned direction as the sampled value of the angle of the PTZ camera; using the sampled value of the angle of the PTZ camera and the sampled value of the pixel coordinates corresponding to the camera as a set of the above sampling data when the PTZ camera is facing one of the aforementioned directions; and obtaining multiple sets of sampling data when the PTZ camera is facing different directions by changing the direction in which the PTZ camera is facing.

[0016] In the above technical solution, when acquiring multiple sets of sampling data, the PTZ camera calculates a homography transformation matrix between the PTZ camera and the camera for each direction it faces. When the direction the PTZ camera faces changes, the homography transformation matrix also changes. This homography transformation matrix can accurately characterize the pixel conversion relationship between the imaging images of the PTZ camera and the camera when the PTZ camera faces a specific direction. Through this homography transformation matrix, it is beneficial to accurately map the pixel coordinates of a specified pixel in the imaging image of the PTZ camera to the imaging image of the camera, obtain accurate mapping coordinates, and thus obtain accurate sampling data, thereby improving the accuracy of the mapping relationship obtained based on the multiple sets of sampling data.

[0017] In combination with the first aspect and the above implementation methods, in some possible implementation methods, the mapping expression for constructing the above mapping relationship includes: determining a first relationship expression between the close-up angle of the PTZ camera and the target angle; wherein the target angle refers to the angle between the principal ray of the camera and the target direction, and at the close-up angle, the optical axis of the PTZ camera coincides with the principal ray of the PTZ camera; determining a second relationship expression between the target angle and the pixel coordinates of the pixels in the image of the camera; and constructing the mapping expression for the above mapping relationship based on the first relationship expression and the second relationship expression.

[0018] In the above technical solution, considering the imaging relationship between the PTZ camera and the camera, the step of constructing the mapping expression is broken down into: constructing a first relational expression and a second relational expression. Since the first relational expression represents the relationship between the close-up angle of the PTZ camera and the target angle, and the second relational expression represents the relationship between the target angle and the pixel coordinates of the pixels in the image of the camera, that is, the first relational expression and the second relational expression have the same parameter, namely the target angle. Therefore, by combining the first relational expression and the second relational expression, it is convenient to construct the mapping expression between the close-up angle of the PTZ camera and the pixel coordinates of the pixels in the image of the camera.

[0019] In conjunction with the first aspect and the above implementation methods, in some possible implementation methods, the close-up angle of the PTZ camera includes: a close-up pitch angle and a close-up horizontal rotation angle; the target angle includes: a first target angle and a second target angle; the target direction includes a first direction and a second direction; the first direction is the optical axis direction of the camera; the second direction is perpendicular to the first direction; the first target angle refers to the angle between the principal ray of the camera and the first direction; the second target angle refers to the angle between the principal ray of the camera and the second direction; determining the first relationship expression between the close-up angle of the PTZ camera and the target angle includes: determining the first relationship expression between the close-up pitch angle of the PTZ camera and the first target angle, and determining the first relationship expression between the close-up horizontal rotation angle of the PTZ camera and the second target angle; determining the second relationship expression between the target angle and the pixel coordinates of the pixels in the image of the camera includes: determining the second relationship expression between the first target angle and the pixel coordinates of the pixels in the image of the camera, and determining the second relationship expression between the second target angle and the pixel coordinates of the pixels in the image of the camera.

[0020] Combining the first aspect and the above implementation methods, in some possible implementation methods, the above mapping relationship includes: a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; the above mapping expression includes: a first mapping expression for the first mapping relationship and a second mapping expression for the second mapping relationship; the sampled values ​​of the angle of the PTZ camera include: sampled values ​​of the pitch angle and sampled values ​​of the horizontal rotation angle; the above method of solving for the calibration values ​​of the parameters to be calibrated based on the above multiple sets of sampled data and the above mapping expression to obtain the above mapping relationship includes: substituting the sampled values ​​of the pitch angle and the sampled values ​​of the pixel coordinates in each set of the above sampled data into the above first mapping expression to solve for the calibration values ​​of the parameters to be calibrated in the above first mapping expression to obtain the above first mapping relationship; substituting the sampled values ​​of the horizontal rotation angle and the sampled values ​​of the pixel coordinates in each set of the above sampled data into the above second mapping expression to solve for the calibration values ​​of the parameters to be calibrated in the above second mapping expression to obtain the above second mapping relationship.

[0021] Secondly, a close-up image output device is provided for use in a remote conferencing system. The remote conferencing system includes a display device and at least one PTZ camera. The device includes: at least one PTZ camera and the display device, wherein the display device is equipped with a camera. The device includes: an acquisition module for acquiring the image captured by the camera; a detection module for detecting the target area of ​​the object to be captured in the image; a determination module for determining the parameter value of the angle control parameter of the PTZ camera based on the target area and a set mapping relationship; wherein the mapping relationship includes the connection between each area in the image captured by the camera and the angle control parameter of the PTZ camera; and a control module for moving the PTZ camera toward the object to be captured according to the parameter value of the angle control parameter, thereby capturing a close-up image of the object.

[0022] Thirdly, a display device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the display device to perform the methods of the first aspect or any possible implementation thereof.

[0023] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0024] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0025] Figure 1 is a schematic diagram of a remote conferencing system provided in an embodiment of this application;

[0026] Figure 2 is a schematic diagram of a panoramic image captured by a camera built into a display device and a close-up image captured by a PTZ camera, according to an embodiment of this application.

[0027] Figure 3 is a schematic diagram of a display interface provided in an embodiment of this application;

[0028] Figure 4 is a schematic flowchart of a close-up shot shooting method provided in an embodiment of this application;

[0029] Figure 5 is a schematic diagram of the imaging relationship between a PTZ camera and a webcam provided in an embodiment of this application;

[0030] Figure 6 is a schematic diagram of another remote conferencing system provided in an embodiment of this application;

[0031] Figure 7 is a schematic flowchart of another method for capturing close-up images provided in an embodiment of this application;

[0032] Figure 8 is a schematic diagram of a close-up image output device provided in an embodiment of this application;

[0033] Figure 9 is a schematic diagram of the structure of a display device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0035] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0036] In video conferencing scenarios, remote video functionality is primarily based on the camera built into the display device. However, due to the inherent hardware characteristics of the camera (such as lens numerical aperture, imaging distortion, and sensor noise), video conferencing functionality has many limitations, including low image resolution, blurred images, and limited imaging distance.

[0037] When conducting multi-person video conferences using display devices, it is often necessary to output close-up shots of a speaker to improve communication efficiency. To achieve this close-up function, common methods currently include cropping the area of ​​interest from the image obtained by the display device's built-in camera, or using an additional digital zoom lens module to image the area of ​​interest. The inventors of this application have discovered that while these methods can improve the imaging quality of close-up shots to some extent, the resulting image quality is still poor. The reason for this poor imaging quality is as follows:

[0038] The above method of extracting the area of ​​interest in a person's image is equivalent to taking a picture of the entire image and then cropping it. If the resolution of the original image is not high enough, the cropped close-up area may appear blurry or pixelated because cropping enlarges each pixel, reduces the number of pixels per inch, and lowers the clarity of details.

[0039] Regarding the above-mentioned method of imaging the area of ​​interest in a human figure using an additional digital zoom lens module, since digital zoom does not magnify the image by physically moving the lens, but by interpolating and magnifying the existing pixels through software algorithms, this magnification method will also lead to the loss of image details, the edges may become blurred, and the overall sharpness will decrease.

[0040] To achieve close-up functionality, another approach is to use lenses with better optical performance, such as giving the display device's built-in camera a larger numerical aperture. However, this has two drawbacks: firstly, it means an increase in camera size, which will affect the appearance or industrial design of the display device itself; secondly, this solution is not suitable for existing display device models and is ineffective for them.

[0041] Based on this, in order to at least solve the aforementioned technical problem of poor imaging effect of close-up images, and to avoid the drawbacks brought about by the larger numerical aperture of the camera of the display device, this application provides a method for shooting close-up images, which is applied to a remote conferencing system, the remote conferencing system including at least one PTZ camera and a display device.

[0042] In this embodiment, the display device is equipped with at least one PTZ camera. As a device with optical zoom capability, the PTZ camera has the advantages of both a global wide viewing angle and a local close-up staring view. By combining the excellent imaging performance of the PTZ camera with the display device, the linkage between the PTZ camera and the camera built into the display device can be flexibly realized. This can significantly improve the close-up imaging effect in conference scenarios, and there is no need to make the camera built into the display device have a larger numerical aperture, so it will not affect the appearance or industrial design of the display device itself, and it can also be applied to existing display device models. The PTZ camera attitude adjustment method involved in this application differs from the traditional manual physical adjustment of the PTZ camera's imaging angle, as well as from the method of using real-time online calculation of feature matching points to obtain pose adjustment parameters to drive the PTZ camera's attitude adjustment. In order to achieve flexible cooperation between the PTZ camera and the display device for imaging conference scenes, this application pre-captures the conference scene using the display device's camera and divides it into multiple local image regions. The mapping relationship between each local image region and the PTZ camera's angle control parameters is pre-calculated and stored. This allows the PTZ camera to automatically cooperate with the display device to perform close-up imaging of any conference participant or local scene during real-time video conferencing, thus ensuring both the presentation quality and efficiency of close-up images.

[0043] In one possible implementation, as shown in Figure 1, the remote conferencing system includes a display device 101 and a PTZ camera 102. The display device 101 and the PTZ camera 102 can be installed in the conferencing environment, and their relative positions are fixed. For example, as shown in Figure 1, the display device 101 is mounted on a wall in the conferencing environment, the camera in the display device 101 is located at point C in the figure, and the PTZ camera 102 is positioned in front of the display device 101. Of course, the PTZ camera 102 can also be positioned in other locations in the conferencing environment, such as above the display device 101 or to the side of the display device 101; this embodiment does not specifically limit this.

[0044] It should be noted that Figure 1 only shows an example of a remote conferencing system with one PTZ camera. In actual implementation, multiple PTZ cameras can be set up. Furthermore, the display device 101 is not limited to being set on the wall of the meeting scene, but can also be set up in other locations in the meeting scene. This embodiment does not make specific limitations in this regard.

[0045] Display device 101 typically refers to a conference terminal with a display screen, specifically an interactive flat panel. This interactive flat panel has a built-in camera that can capture the conference scene in the video conference and output an image of the conference scene, which is usually a panoramic view of the conference scene.

[0046] PTZ camera 102 typically refers to a camcorder with PTZOOM (Pan-Tilt-Zoom) functionality, featuring lenses with zoom, tilt, and pan capabilities. Zoom allows adjustment of the lens's focal length within a certain range, enabling clear shooting of both close-up and distant subjects. Tilt adjusts the vertical angle, moving the lens up and down. Pan rotates the lens horizontally, expanding the lateral field of view and covering a wider area.

[0047] The PTZ camera 102 can be connected to the display device 101. The display device 101 can flexibly control the movement of the PTZ camera 102 to output close-up images of a specified close-up area.

[0048] The field of view captured by the camera built into display device 101 is larger than that captured by PTZ camera 102. The camera built into display device 101 is used to capture and output a panoramic view of the meeting scene, while PTZ camera 102 is used to capture and output close-up views of objects to be captured in the meeting scene.

[0049] Figure 2 is a schematic diagram of a panoramic view captured by the camera built into the display device and a close-up view captured by the PTZ camera in an embodiment of this application. As can be seen from Figure 2, the panoramic view 201 captured by the camera built into the display device includes 7 participants in the meeting scene, while the close-up view 202 captured by the PTZ camera only shows 1 of the 7 participants, and this 1 participant is the subject to be shown in the close-up.

[0050] Understandably, remote conferencing typically involves an initiator and a receiver. The initiator's meeting environment is equipped with a display device and a PTZ camera, while the receiver's meeting environment also has a display device and a PTZ camera. For ease of description, the display device and PTZ camera in the initiator's meeting environment will be referred to as the initiator's display device and initiator's PTZ camera, respectively, and the display device and PTZ camera in the receiver's meeting environment will be referred to as the receiver's display device and receiver's PTZ camera, respectively.

[0051] In one possible implementation, a close-up view of the meeting scene where the initiator is located, captured by the initiator's PTZ camera, can be output to the receiver's display device and / or the initiator's display device for display.

[0052] In one possible implementation, a close-up view of the meeting scene where the receiver is located, captured by the receiver's PTZ camera, can be output to the receiver's display device and / or the initiator's display device for display.

[0053] In one possible implementation, the receiving display device can display a panoramic image captured by its own camera, and / or display a panoramic image captured by the initiating display device's own camera. For example, the initiating display device sends a panoramic image captured by its own camera to the receiving display device for display.

[0054] In one possible implementation, the initiating display device can display a panoramic image captured by its built-in camera, and / or display a panoramic image captured by the receiving display device's built-in camera. For example, the receiving display device sends a panoramic image captured by its built-in camera to the initiating display device for display.

[0055] In the specific implementation, the display interface of the receiving or initiating display device can select the screen according to actual needs. Users can choose one or more of the following screens: Screen 1 - a panoramic view captured by the receiving display device's built-in camera; Screen 2 - a panoramic view captured by the initiating display device's built-in camera; Screen 3 - a close-up view captured by the receiving PTZ camera; Screen 4 - a close-up view captured by the initiating PTZ camera. If one screen is selected, the display interface will show it in one display area; if two screens are selected, they will be shown in two separate display areas; if three screens are selected, they will be shown in three separate display areas, and so on.

[0056] Referring to Figure 3, assuming the initiator selects to display screens 2 and 3 on the initiator's display device via the screen selection button, a schematic diagram of the initiator's display device's display interface can be seen in Figure 3. The display interface 300 includes a first display area 301 and a second display area 302. The first display area 301 is used to display screen 2, and the second display area 302 is used to display screen 3. Optionally, the display area occupied by the first display area 301 and the second display area 302 on the display interface can be set by the initiator according to actual needs and can be adjusted in real time during the meeting.

[0057] Figure 4 is a schematic flowchart of a close-up shot shooting method provided in an embodiment of this application.

[0058] For example, as shown in Figure 4, the method includes:

[0059] Step 401: Acquire the image captured by the camera.

[0060] Step 402: Detect the target area where the object to be zoomed in is located in the image.

[0061] Step 403: Determine the target parameter values ​​of the PTZ camera's angle control parameters based on the target area and the established mapping relationship; wherein, the mapping relationship includes the mapping relationship between each area in the camera's imaging image and the parameter values ​​of the PTZ camera's angle control parameters.

[0062] Step 404: Based on the target parameter value of the angle control parameters, make the PTZ camera move towards the object to be zoomed in and capture a close-up image of the object.

[0063] In the embodiment shown in Figure 4, the display device and the PTZ camera work together. The PTZ camera can automatically track and output a close-up image of the object to be zoomed in on. By acquiring the image captured by the camera in the display device, the target area where the object to be zoomed in on is located in the image is detected. Based on the target area and the set mapping relationship, the target parameter value of the PTZ camera's angle control parameters is determined. That is, in this embodiment, even if the PTZ camera has not captured the object to be zoomed in on, the target parameter value of the PTZ camera's angle control parameters can be solved. This target parameter value can guide the PTZ camera to move towards the object to be zoomed in on, which is beneficial for the PTZ camera to accurately capture a close-up image of the object to be zoomed in on, thereby improving the imaging effect of the close-up image. Furthermore, since the aforementioned mapping relationship includes the connection between each region in the camera's image and the parameter values ​​of the PTZ camera's angle control parameters, based on this mapping relationship and the target region, the target parameter values ​​of the angle control parameters that enable the PTZ camera to move toward the object to be captured can be directly obtained. This eliminates the need to calculate the angle that the PTZ camera needs to rotate to capture the object after it has been photographed by the camera. This saves the computing power and time resources required for the PTZ camera to first capture the object and then calculate the required rotation angle through feature matching. This facilitates the rapid and efficient acquisition of the target parameter values ​​of the PTZ camera's angle control parameters, thereby improving the output efficiency and accuracy of the close-up images and ensuring the system's immediate response and efficient operation.

[0064] In this embodiment, the excellent imaging performance of the PTZ camera is combined with the display device, which significantly improves the imaging effect of close-up shots in existing conference scenarios. Its main features include the following aspects:

[0065] Firstly, the PTZ camera is structurally completely independent of the display device, allowing it to adapt to the output needs of various existing display devices for close-up images in meetings without changing the structure of the display device.

[0066] Secondly, the display devices that can be used with the PTZ camera include, but are not limited to, existing models and possible future models, which improves the applicability of the close-up shooting method in this embodiment.

[0067] Thirdly, the hardware parameters and installation location of the PTZ camera can be flexibly designed according to different imaging needs, without being limited by the structure of the display device, and the number of PTZ cameras can be adjusted according to actual needs.

[0068] Fourthly, the PTZ camera itself has horizontal and vertical rotation functions, and the lens itself can be equipped with optical or digital zoom functions, which is conducive to accurately pointing towards the subject to be zoomed in, so as to achieve accurate imaging of the subject to be zoomed in.

[0069] The specific implementation of each step in the embodiment shown in Figure 4 is explained below:

[0070] In step 401, the display device can acquire the image captured by its built-in camera. The image can be a panoramic view of the meeting scene on which the display device is installed, such as panoramic view 201 as shown in Figure 2.

[0071] For example, after receiving a command to start a remote conference and a command to start the camera, the display device can control its built-in camera to start so as to capture a panoramic view of the conference scene.

[0072] In step 402, the object to be highlighted is the object that needs to be highlighted in the remote meeting, that is, the object that we want to capture and highlight through the PTZ camera.

[0073] Optionally, the subject to be zoomed in on may be a participant who is speaking, or a participant with preset target characteristics, such as specific facial expressions, specific gestures, or other nonverbal information.

[0074] Optionally, the subject to be highlighted can also be a participant designated by the meeting host or other participants with selection authority. For example, the meeting host can designate a participant as the subject to be highlighted via voice command, or select a participant from among the participants displayed on the display device via touch operation.

[0075] It should be noted that the above example only uses a participant as the object to be highlighted. In actual implementation, the object to be highlighted can also be other non-human objects besides the participant, such as an object to be displayed in the meeting scene, a designated display area in the meeting scene, a PPT to be presented, a blackboard with handwritten content, etc.

[0076] Specifically, the display device can first detect the object to be zoomed in on in the image, and then use the area where the object to be zoomed in is located in the image as the target area.

[0077] In one possible implementation, the display device can process the image captured by the camera to obtain the location distribution data of the attendees. The display device can also use an array microphone to collect sound information to be located in real time, where the sound information is the voice of the attendees. Then, based on voice tracking technology, the location of the sound information is determined, thus obtaining the sound's location information.

[0078] The location information of the sound is matched with the location information of each participant in the participant location distribution data to identify those whose location information matches the sound as the objects to be zoomed in on. Next, a target detection algorithm is used to determine the bounding box containing the object to be zoomed in on within the image, and the area containing this bounding box is designated as the target region. In this example, since the object to be zoomed in on is a participant, the bounding box could be the participant's face bounding box.

[0079] In one possible implementation, when the object to be zoomed in on is a participant, the target area can be the center point (x, y) of the face detection box of the object to be zoomed in on.

[0080] For example, a face detection bounding box for the subject to be zoomed in on can be extracted from the image based on lip movement recognition technology. For instance, based on the image, the lip movement area is determined, and based on a preset expansion ratio, the lip movement area is expanded outward by a certain number of pixels to select the entire face area, ensuring that the face detection bounding box can completely cover the speaker's, i.e., the face of the subject to be zoomed in on.

[0081] For example, a face detection bounding box for the object to be zoomed in on can be extracted from the image based on sound source localization technology. For instance, a Direction of Arrival (DOA) algorithm can be applied to process the signals collected by the microphone array, estimate the azimuth and elevation angles of the sound source, and transform the azimuth and elevation angles of the sound source in three-dimensional space to the two-dimensional coordinate system of the image, so as to determine the area corresponding to the position of the sound source in the image as the face detection bounding box.

[0082] For example, face detection technology can be used to extract the face detection bounding box of the object to be zoomed in on from the image. For instance, face recognition technology can be used to obtain various face regions present in the image, and from these face regions, the speaker's face region can be obtained as the face detection bounding box of the object to be zoomed in on.

[0083] In practical implementation, two or three of the above-mentioned lip movement recognition technology, sound source localization technology and face detection technology can be combined to extract the face detection box of the object to be zoomed in on from the image.

[0084] The following is a detailed explanation of an implementation method combining the aforementioned lip movement recognition, sound source localization, and face detection technologies: First, a general face detection method is used to quickly detect all face regions in the entire image. Then, based on the sound source localization results, the face region pointed to by the sound source localization result is determined from all detected face regions in the entire image. Next, based on the lip movement recognition technology, the face region most likely to be the one speaking is selected from the face region pointed to by the sound source localization result as the face detection box for the subject to be zoomed in on.

[0085] Taking the combination of the aforementioned lip movement recognition technology, sound source localization technology, and face detection technology as an example, another implementation method is explained in detail: First, the sound signal collected by the array microphone is acquired, and the sound signal is processed based on the sound source localization technology to determine the location result of the sound source area; face detection technology is used to perform face detection on the entire imaging screen to obtain the location result of the face detection; the location result of the sound source area is matched with the location result of the face detection, and the matched location result is determined as the location of the area to be zoomed in on; when there are two or more faces in the location of the area to be zoomed in on, the face area in the location of the area to be zoomed in on is identified based on the lip movement recognition technology, and the face area most likely to be the face that is speaking is selected as the face detection box of the object to be zoomed in on.

[0086] In an exemplary embodiment, prior to step 402, the method further includes receiving a close-up view request for the object to be captured. That is, step 402 is only executed upon receiving a close-up view request for the object to be captured, ensuring that the detection of the object meets the presentation requirements and satisfies the user's personalized needs during the remote meeting process. The user can choose whether to capture a close-up view based on actual needs.

[0087] Specifically, the conference interface of the display device may display virtual buttons, which are used to trigger a request to display a close-up image. If a user wishes to capture and display a close-up image, they can do so by clicking the virtual button. When the display device detects that the virtual button has been clicked, it can determine that it has received a request to display a close-up image of the object to be captured, and thus execute step 402.

[0088] Optionally, users can also trigger the close-up display request through air gestures, voice commands, or other means. When the display device detects an air gesture or voice command used to trigger the close-up display request, it confirms that it has received the close-up display request.

[0089] In step 403, the mapping relationship can be stored in the display device. Since the mapping relationship includes the connection between each region in the camera's image and the parameter values ​​of the PTZ camera's angle control parameters—that is, each region in the image corresponds to its own angle control parameter value—the display device can use the parameter value of the angle control parameter corresponding to the target region as the target parameter value based on this mapping relationship. This target parameter value is used to instruct the PTZ camera to move towards the object to be zoomed in on, so that the center of the PTZ camera can be aligned with the object to be zoomed in on. This target parameter value can be understood as: the target angle value of the PTZ camera, that is, the angle value that the PTZ camera is expected to achieve.

[0090] For example, the above mapping relationship can include the connection between each region in the camera's image and the parameter values ​​of the PTZ camera's angle control parameters when taking close-up shots of each region; that is, the connection between each region in the camera's image and the actual angle values ​​of the PTZ camera when taking close-up shots of each region. Taking a close-up shot of a region in the camera's image means that the center of the PTZ camera is aligned with the physical space region corresponding to that region in the image, i.e., the center of the PTZ camera is aligned with the object to be photographed within the physical space region. Specifically, calibration experiments can be conducted in advance in a meeting scene where the relative positions of the camera and the PTZ camera are fixed to obtain the above mapping relationship. For example, the meeting scene image can be captured in advance using the camera of the display device and divided into multiple local image regions. The angle control parameters of the PTZ camera when taking close-up shots of the physical space region corresponding to each local image region can be determined in advance to obtain the mapping relationship between each local image region and the PTZ camera's angle control parameters, and this mapping relationship can be stored.

[0091] In step 404, the display device can send target parameter values ​​to the PTZ camera based on the communication connection with the PTZ camera. This causes the PTZ camera to move towards the object to be zoomed in on in the physical space area according to the target parameter values, until the PTZ camera is directly facing the object. Then, it takes a picture of the object in the physical space area and sends the close-up image of the object to the display device. Here, "the PTZ camera moving directly facing the object" can be understood as the PTZ camera moving until its center is directly facing the center of the object.

[0092] Assuming that the display device in steps 401 to 404 above is the initiating display device, and the corresponding PTZ camera is also the initiating PTZ camera, then after receiving the close-up image sent by the initiating PTZ camera, the initiating display device can choose to display or not display the close-up image on its own device according to actual needs, or it can send the close-up image to the receiving display device to display the close-up image on the receiving display device.

[0093] Assuming that the display device in steps 401 to 404 above is a receiving display device, and the corresponding PTZ camera is also a receiving PTZ camera, then after receiving the close-up image sent by the receiving PTZ camera, the receiving display device can choose to display or not display the close-up image on the receiving display device according to actual needs, or it can send the close-up image to the initiating display device to achieve the display of the close-up image on the initiating display device.

[0094] In one possible implementation, multiple PTZ cameras are set up in the conference scenario, and the relative position of each PTZ camera to the display device is fixed. In this case, the mapping relationship can include the mapping relationship corresponding to each PTZ camera. In step 403, the target parameter value of the angle control parameter of each PTZ camera can be determined according to the target area and the mapping relationship corresponding to each PTZ camera. The display device can communicate with each PTZ camera to obtain the current actual angle value of each PTZ camera. Based on the difference between the target parameter value of the angle control parameter of each PTZ camera (i.e., the target angle value) and the current actual angle value of each PTZ camera, the PTZ camera with the smallest difference is selected as the target PTZ camera. Therefore, in step 404, the target PTZ camera can be controlled to move towards the object to be zoomed in on according to the target parameter value of the angle control parameter of the target PTZ camera, so that the target PTZ camera outputs a close-up image of the object to be zoomed in on. Since the difference between the target angle value and its current actual angle value of the target PTZ camera is minimal, the amount of movement required to control the target PTZ camera to move toward the object to be zoomed in is small, which is beneficial for quickly aligning with the object to be zoomed in and thus quickly outputting a close-up image of the object to be zoomed in.

[0095] In one possible implementation, the mapping relationship described above specifically refers to the connection between the pixel coordinates of each region in the camera's image and the parameter values ​​of the PTZ camera's angle control parameters. This mapping relationship can also be expressed as the connection between the pixel coordinates of each pixel in the camera's image and the parameter values ​​of the PTZ camera's angle control parameters. For example, this mapping relationship can be represented as: v = f(x, y); where v represents the angle control parameters, x represents the horizontal pixel coordinate of a pixel in the camera's image, and y represents the vertical pixel coordinate of a pixel.

[0096] One implementation of step 403 above may include the following steps S11 to S12:

[0097] S11: Determine the target pixel coordinates of the target area in the camera's image.

[0098] S12: Substitute the target pixel coordinates into the mapping relationship to obtain the target parameter value of the angle control parameters of the PTZ camera.

[0099] Specifically, the display device can use the pixel coordinates of the center pixel of the target area in the image as the target pixel coordinates (x, y, y). i y iIt is understandable that the center pixel is located at the exact center of the target area, and the center pixel corresponds to the exact center of the object to be zoomed in on. Based on the pixel coordinates of the center pixel, the target parameter values ​​obtained by mapping are beneficial to enabling the PTZ camera to move to the exact center of the object to be zoomed in on, thereby further improving the imaging effect of the close-up image.

[0100] Furthermore, through the mapping relationship v = f(x, y) mentioned above, it can be seen that when the values ​​of x and y are known, the target parameter value of the angle control parameter can be obtained. Combining the above example, the target pixel coordinates (x, y) can be calculated. i y i Substitute the above mapping relationship into the value of v and calculate the value of v. Use the value of v as the target parameter value.

[0101] In this embodiment, by establishing a relationship between the pixel coordinates of each region in the camera's image and the angle control parameters of the PTZ camera, it is convenient to directly calculate the target parameter value by substituting the target pixel coordinates in the target region into the mapping relationship. This allows the PTZ camera to output close-up images of the region represented by the target pixel coordinates. Furthermore, the angle control parameters of the PTZ camera are essentially mechanical motion control parameters. By mapping intuitive pixel coordinates to complex mechanical motion control parameters, the intelligence level of the remote conferencing system is greatly improved.

[0102] In an exemplary embodiment, the angle control parameters include: pitch angle Tilt and horizontal rotation angle Pan, and the mapping relationship includes a first mapping relationship corresponding to the pitch angle Tilt and a second mapping relationship corresponding to the horizontal rotation angle Pan. The implementation of S12 includes the following S121 to S122:

[0103] S121: Substitute the target pixel coordinates into the first mapping relationship to obtain the target parameter value of the PTZ camera's pitch angle Tilt.

[0104] S122: Substitute the target pixel coordinates into the second mapping relationship to obtain the target parameter value of the horizontal rotation angle Pan of the PTZ camera.

[0105] The first mapping relationship mentioned above can be represented as: tilt i = f1(x, y), where tilt i Indicates the pitch angle, and sets the target pixel coordinates (x, y, y) to y. i y i Substituting the first mapping relationship mentioned above, we can calculate tilt. i The value of tilt will be determined by tilt. i The value of is used as the target parameter value for the pitch angle. Here, i represents the identifier of the center pixel of the target region.

[0106] The second mapping relationship mentioned above can be represented as: pan i =f2(x, y), where pan i This indicates the horizontal rotation angle, which represents the target pixel coordinates (x, y). i y i Substituting the second mapping relationship mentioned above, we can calculate pan. i The value of pan i The value of is used as the target parameter value for the horizontal rotation angle.

[0107] If the implementation of S12 includes S121 to S122, the implementation of step 404 may include: based on the target parameter value tilt of the PTZ camera. i Control the actual pitch angle of the PTZ camera, and based on the target parameter value of the horizontal rotation angle of the PTZ camera, pan... i The actual pitch angle of the PTZ camera is controlled to move the PTZ camera towards the subject to be zoomed in on. When the PTZ camera moves to be directly facing the subject, the actual pitch angle of the PTZ camera is tilt. i And the actual horizontal rotation angle of the PTZ camera is pan. i .

[0108] Assume, tilt i It is 20°, and pan i If the angle is 10°, then control the movement of the PTZ camera so that the actual pitch angle of the PTZ camera reaches 20° and the actual horizontal rotation angle reaches 10°, thereby enabling the PTZ camera to move directly to the subject being viewed. Specifically, the center of the PTZ camera can be aligned with the center of the subject being viewed.

[0109] In this embodiment, corresponding first and second mapping relationships are set for the pitch angle and horizontal rotation angle of the PTZ camera, respectively. This facilitates the rapid and accurate acquisition of the target parameter value of the pitch angle through the first mapping relationship, and the rapid and accurate acquisition of the target parameter value of the horizontal rotation angle through the second mapping relationship. Furthermore, based on these target parameter values ​​for the pitch angle and horizontal rotation angle, the PTZ camera is controlled to move towards the object to be zoomed in on. This allows for precise control of the PTZ camera in both horizontal and vertical dimensions, enabling rapid positioning of the PTZ camera to align with the object, thus improving the efficiency and accuracy of the output close-up image of the object.

[0110] In an exemplary embodiment, the method for determining the above mapping relationship may include the following steps S21 to S23:

[0111] S21: Construct the mapping expression for the mapping relationship. The mapping expression includes the parameters to be calibrated.

[0112] Specifically, the relative positional relationship of the combined system consisting of the PTZ camera and the camera can be modeled, and the imaging relationship between the PTZ camera and the camera can be determined based on the relative positional relationship and the optical imaging principle. Based on this imaging relationship, the mapping expression of the above mapping relationship can be derived.

[0113] In some embodiments, the implementation of S21 may include the following S211 to S213:

[0114] S211: Determine the first relational expression between the close-up angle of the PTZ camera and the angle between the target and the target.

[0115] The aforementioned target angle refers to the angle between the camera's principal ray and the target direction. The camera's principal ray is the ray that passes through the center of the lens's pupil in an optical system. The pupil is the actual light-gathering aperture of the lens. The extension of the principal ray helps determine the image position of the object on the imaging plane. The camera's principal ray can be understood as the line connecting the optical center of the camera and the image point of the target point on the camera's imaging plane; that is, the line connecting the optical center and the corresponding image point of the target point. The target direction can be the optical axis or a direction perpendicular to the optical axis.

[0116] The target point can be a key feature point of the object to be photographed. This key feature point can be the center point of the object, the location of the light source, or the focal point of reflected light from the object. If the object is a person, the target point can be the tip of the person's nose. If the object is an object, the target point can be the center point of the object's surface. The selection of this target point can be determined according to the shooting requirements or the properties of the object itself, and this application does not impose specific limitations.

[0117] The aforementioned close-up angle refers to the actual angle at which the optical axis of the PTZ camera coincides with the principal ray of the PTZ camera when taking a close-up of the subject. In other words, at the close-up angle, the optical axis and principal ray of the PTZ camera coincide. Ideally, when the PTZ camera takes a close-up of the subject, the optical axis and principal ray of the PTZ camera should coincide. The principal ray of the PTZ camera can be understood as the line connecting the optical center of the PTZ camera to the image point of the subject on the PTZ camera's imaging plane, where the image point of the subject is the center point of the PTZ camera's imaging plane.

[0118] Based on the imaging relationship between the PTZ camera and the camera lens, it can be determined that there is a one-to-one correspondence between the close-up angle of the PTZ camera and the angle between the target and the target. Based on this, the first relationship expression between the close-up angle of the PTZ camera and the angle between the target and the target can be derived.

[0119] In one possible implementation, the close-up angle of the PTZ camera includes: a close-up pitch angle and a close-up horizontal rotation angle. The close-up pitch angle and the close-up horizontal rotation angle refer to the actual pitch angle and the actual horizontal rotation angle of the PTZ camera when the optical axis of the PTZ camera coincides with the principal ray of the PTZ camera during a close-up of an object. The target angle includes: a first target angle and a second target angle. The target direction includes a first direction and a second direction. The first direction is the optical axis direction of the camera, and the second direction is perpendicular to the first direction. The first target angle refers to the angle between the principal ray of the camera and the first direction, and the second target angle refers to the angle between the principal ray of the camera and the second direction. When the optical axis direction of the camera is horizontal, the first direction is horizontal, and the second direction is vertical. Correspondingly, the implementation of S211 includes:

[0120] Determine the first relational expression between the close-up pitch angle of the PTZ camera and the angle between the first target, and determine the first relational expression between the close-up horizontal rotation angle of the PTZ camera and the angle between the second target.

[0121] To facilitate understanding, the first relationship expression for determining the close-up pitch angle of the PTZ camera and the angle with the first target is explained below with reference to Figure 5:

[0122] Figure 5 is a schematic diagram of the imaging relationship between the PTZ camera and the webcam. In Figure 5, assuming the object being imaged, i.e., the object to be photographed, is located at the position of the pentagram in the diagram, the relative positional relationship between the PTZ camera and the webcam is shown. In the diagram, it is assumed that the optical axis of the webcam is horizontal, and the principal ray of the webcam makes an angle θ with the optical axis, which is the aforementioned first target angle. In Figure 5, the principal ray of the webcam is the line connecting the optical center of the webcam and the image point of the target point of the object in Figure 5 on the imaging plane of the webcam. When the PTZ camera takes a close-up of the object, ideally, the optical axis of the PTZ camera coincides with the direction of the principal ray, and the angle between it and the horizontal direction is α. In Figure 5, the principal ray of the PTZ camera is the line connecting the optical center of the PTZ camera and the image point of the target point of the object in Figure 5 on the imaging plane of the PTZ camera, and the image point of this target point on the imaging plane of the PTZ camera is the center point of the imaging plane of the PTZ camera.

[0123] Obviously, if the initial orientation of the PTZ camera is exactly horizontal, the close-up pitch angle is exactly equal to the angle α. According to geometric relationships, there is a one-to-one correspondence between the close-up pitch angle α and the first target angle θ. The first relationship expression between the close-up pitch angle α and the first target angle θ can be expressed as the following formula (1):

[0124] Formula (1) indicates that the angle θ between the main ray of the camera and the horizontal direction corresponds one-to-one with the close-up pitch angle α of the PTZ camera when taking a close-up of the object in the image. From a geometrical perspective, the correspondence between α and θ is non-linear.

[0125] It is understandable that the principle of determining the first relationship expression between the close-up horizontal rotation angle of the PTZ camera and the second target angle is similar to the principle of determining the above formula (1), only the horizontal direction in Figure 5 is replaced with the vertical direction. Assuming that the optical axis of the camera is along the horizontal direction, the principal ray of the camera forms a second target angle with the vertical direction, and this second target angle is denoted as angle β. When the PTZ camera takes a close-up of the above object, ideally, the optical axis of the PTZ camera coincides with the direction of the principal ray, and its angle with the vertical direction is denoted as angle γ. If the initial orientation of the PTZ camera is exactly in the vertical direction, the close-up horizontal rotation angle is exactly equal to angle γ. According to the geometric relationship, there is a one-to-one correspondence between the close-up horizontal rotation angle γ and the second target angle β, and the first relationship expression between the close-up horizontal rotation angle γ and the second target angle β can be the following formula (2):

[0126] Formula (2) indicates that the angle β between the main ray of the camera and the vertical direction corresponds one-to-one with the horizontal rotation angle γ of the PTZ camera when taking a close-up of the object in the image. From a geometrical perspective, the correspondence between β and γ is non-linear.

[0127] S212: Determine the second relational expression between the target angle and the pixel coordinates of the pixels in the camera's image.

[0128] It is understandable that, since the target angle refers to the angle between the camera's principal ray and the target direction, there will be a one-to-one correspondence between this target angle and the pixel coordinates of the pixels in the camera's image. Based on this, a second relational expression between the target angle and the pixel coordinates of the pixels in the camera's image can be constructed. Specifically, this second relational expression can be derived based on the camera's imaging model, combined with the camera's intrinsic and extrinsic parameter matrices and distortion parameters.

[0129] In this embodiment, although the imaging model of the camera, the intrinsic and extrinsic parameter matrix of the camera, and the distortion parameters are involved, it is only for the purpose of deriving the second relational expression. It is not necessary to obtain the specific calibration values ​​of the intrinsic and extrinsic parameter matrix and the distortion parameters, nor is it necessary to consider the specific type of imaging model to which the camera belongs. It can choose any one of the imaging models such as pinhole model, omnidirectional camera model, and multi-view geometric model to derive the above-mentioned second relational expression.

[0130] In one possible implementation, considering that the target angle includes the first target angle and the second target angle mentioned above, the implementation of S212 includes: determining a second relational expression between the first target angle and the pixel coordinates of the pixels in the image of the camera, and determining a second relational expression between the second target angle and the pixel coordinates of the pixels in the image of the camera.

[0131] Specifically, formula (3) below represents the second relationship expression between the first target angle θ and the pixel coordinates (x, y) of the pixel in the camera's image. Formula (4) below represents the second relationship expression between the second target angle β and the pixel coordinates (x, y) of the pixel in the camera's image. θ=F1(x, y) Formula (3) β=F2(x, y) Formula (4)

[0132] S213: Construct a mapping expression for the mapping relationship based on the first relational expression and the second relational expression.

[0133] Specifically, the second relational expression can be substituted into the first relational expression to obtain the mapping expression of the mapping relationship.

[0134] In one possible implementation, the second relationship expression between the first target angle and the pixel coordinates of the pixels in the camera's image can be substituted into the first relationship expression between the PTZ camera's close-up pitch angle and the first target angle to obtain the first mapping expression of the first mapping relationship as shown in formula (5). That is, by substituting the above formula (3) into the above formula (1), formula (5) is obtained.

[0135] In one possible implementation, the second relational expression between the second target angle and the pixel coordinates of the pixels in the camera's image can be substituted into the first relational expression between the close-up horizontal rotation angle of the PTZ camera and the second target angle to obtain the second mapping expression of the second mapping relationship as shown in formula (6). That is, by substituting the above formula (4) into the above formula (2), formula (6) is obtained.

[0136] Expressing the above formulas (5) and (6) in specific mathematical processes, we can conclude that both formulas (5) and (6) can be approximated as quadric surfaces. That is, formula (5) can be expressed as formula (7) and formula (6) can be expressed as formula (8): α≈a t x 2 +b t y 2 +c t xy+d t x+e t y+f t Formula (7) β≈a p x 2 +b p y 2 +c p xy+d p x+e p y+f p Formula (8)

[0137] Among them, a t b t c t d t e t f t For the parameters to be calibrated included in the first mapping expression, a p b p c p d p e p f p These are the parameters to be calibrated included in the second mapping expression. These parameters are essentially unknowns and need to be solved before they can be used to calculate the target parameter values ​​in subsequent use. To solve for the calibration values ​​of these parameters, step S22 is then executed.

[0138] S22: Obtain multiple sets of sampled data.

[0139] Each set of sampling data includes: the sampled angle of the PTZ camera and the sampled pixel coordinates of the camera when the PTZ camera is facing one direction. The angle and pixel coordinates in each set of sampling data are obtained when the PTZ camera is facing one direction.

[0140] The angle sampling value of a PTZ camera refers to the actual angle value of the PTZ camera when it is facing a direction and taking a close-up of an object in that direction.

[0141] The sampled pixel coordinates refer to the mapped coordinates of a specified pixel in the PTZ camera's image when the PTZ camera is facing the aforementioned direction, onto the image of the camera. The specified pixel can be the middle pixel in the PTZ camera's image.

[0142] Specifically, multiple sets of sampling data can be obtained by sampling the pixel coordinates of objects at different positions in the scene where the PTZ camera and the webcam are installed, as well as the actual angle value when the PTZ camera takes a close-up of the object.

[0143] For example, when the PTZ camera takes close-up shots of different objects in the above scene, the PTZ camera faces different directions. Based on this, the actual angle value of the PTZ camera facing different directions can be recorded, and the actual angle value can be used as the angle sampling value. At the same time, the pixel coordinates of the center pixel in the PTZ camera's imaging image can be located and mapped to the imaging image of the camera, and the mapped coordinates can be used as the pixel coordinate sampling value.

[0144] For example, the sampled values ​​of the above angles include: the sampled values ​​of the pitch angle and the sampled values ​​of the horizontal rotation angle. The above recording of the actual angle values ​​when the PTZ camera is facing different directions may include: recording the actual pitch angle value and the actual horizontal rotation angle value when the PTZ camera is facing different directions, using the recorded actual pitch angle value as the sampled value of the pitch angle, and using the recorded actual horizontal rotation angle value as the sampled value of the horizontal rotation angle.

[0145] In the specific implementation, each set of sampled data can be denoted as (pan). c ,tilt c x c y c ), where pan c This refers to the sampled value of the horizontal rotation angle, tilt. c This refers to the sampled value of the pitch angle, (x c y c () refers to the sampled value of the pixel coordinates. The meaning of this set of sampled data is: when the actual horizontal rotation angle of the PTZ camera is tilt... c And the actual pitch angle value is pan c At that time, the PTZ camera can capture the pixel coordinates (x, y) of the image from the camera. c y c Close-up shots of objects within the specified area.

[0146] In one possible implementation, S22 is implemented as follows: S221 to S225:

[0147] S221: For each direction the PTZ camera faces, acquire the image from the PTZ camera and the image from the camera, and perform feature point matching on the image from the PTZ camera and the image from the camera to calculate the homography transformation matrix between the PTZ camera and the camera when the PTZ camera faces the aforementioned direction.

[0148] Specifically, each time the PTZ camera faces a different direction, the display device can acquire the currently captured image from the PTZ camera (hereinafter referred to as the first image) and the currently captured image from the camera (hereinafter referred to as the second image). Then, image features are extracted from the first and second image images respectively, and the homography transformation matrix between the PTZ camera and the camera is calculated through feature matching. Each time the PTZ camera changes its orientation, a homography transformation matrix is ​​calculated using the method described in S221 above, ultimately resulting in multiple homography transformation matrices, each corresponding to a direction in which the PTZ camera is facing.

[0149] For example, the method for calculating the homography transformation matrix corresponding to each direction the PTZ camera faces, based on the first and second image images acquired, may include the following steps S31 to 34:

[0150] S31: Feature matching, which involves finding matching feature points in the first and second imaging frames. This can typically be accomplished using feature detection algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented Fast and Rotated BRIEF).

[0151] S32: Coordinate extraction, that is, extracting the pixel coordinates of these matched feature points in the first and second imaging frames respectively.

[0152] S33: Construct a system of equations, that is, use the basic formula for the homography transformation matrix to construct a system of equations. The constructed system of equations can take the following form:

[0153] Among them, (c x c y (x, y) represents the pixel coordinates of the matched feature point in the first image, and (x, y) represents the pixel coordinates of the matched feature point in the second image.

[0154] S34: Solve the system of equations. Since the homography matrix has 8 degrees of freedom, at least 4 pairs of matching points are needed to uniquely determine a homography matrix. Substitute the pixel coordinates of at least 4 pairs of matching points into the system of equations and solve using the least squares method or other optimization methods to obtain the homography matrix H.

[0155] S222: Based on the homography transformation matrix between the PTZ camera and the camera when the PTZ camera is facing the above direction, the pixel coordinates of the specified pixel in the image of the PTZ camera are mapped to the image of the camera to obtain the mapped coordinates, and the mapped coordinates are used as the sampled values ​​of the corresponding pixel coordinates of the camera.

[0156] Specifically, the specified pixel can be selected from the PTZ camera's image frame as needed. In this embodiment, the center pixel in the PTZ camera's image frame can be used as the specified pixel. Then, using the homography transformation matrix H calculated above for that direction, the pixel coordinates of the center pixel in the PTZ camera's image frame are transformed to obtain the mapped coordinates of the center pixel onto the camera's image frame. This is equivalent to obtaining a one-to-one correspondence between the pixel coordinates of the center pixel in the PTZ camera's image frame and the pixel coordinates in the camera's image frame. At this time, each set of sampled data (pan...) c ,tilt c x c y c The meaning of ) is: when the actual horizontal rotation angle value of the PTZ camera is tilt c And the actual pitch angle value is pan c At that time, the PTZ camera can capture the pixel coordinates (x, y) of the image from the camera. c y c The method involves taking close-up shots of objects within a specified area, ensuring that the center point of the object is precisely located within the center pixel region of the PTZ camera's image. By solving for the mapping relationship using multiple sets of sampled data, a precise mapping between the target area and the target parameter value can be achieved. This ensures that when the PTZ camera is moved based on the target parameter value, the PTZ camera's center can be precisely aligned with the center of the object to be captured, further improving the imaging effect of the close-up image.

[0157] S223: Use the angle value of the PTZ camera when it faces the above-mentioned direction as the sample value of the angle of the PTZ camera.

[0158] Specifically, when the PTZ camera is facing the above-mentioned directions, the actual pitch angle and the actual horizontal rotation angle of the PTZ camera can be recorded respectively. The recorded actual pitch angle value is used as the sample value of the pitch angle, and the recorded actual horizontal rotation angle value is used as the sample value of the horizontal rotation angle.

[0159] S224: The sampled values ​​of the angle of the PTZ camera and the sampled values ​​of the corresponding pixel coordinates of the camera are used as a set of sampled data when the PTZ camera is facing a certain direction.

[0160] As mentioned above, a set of sampled data can be represented as: (pan c ,tilt c x c y c ).

[0161] S225: By changing the orientation of the PTZ camera, multiple sets of sampling data can be obtained when the PTZ camera is facing different directions.

[0162] It is understandable that the PTZ camera can acquire a set of the above-mentioned sampling data for each direction it faces, and finally obtain a set of sampling data corresponding to each direction, forming the above-mentioned multiple sets of sampling data.

[0163] S23: Based on multiple sets of sampled data and mapping expressions, solve for the calibration values ​​of the parameters to be calibrated to obtain the above mapping relationship.

[0164] It is understandable that the mapping expression includes unknown parameters to be calibrated, pixel coordinates of the camera's image, and angle control parameters of the PTZ camera. Each set of sampled data includes sampled values ​​of pixel coordinates and sampled values ​​of the PTZ camera's angle. Therefore, multiple sets of sampled data can be substituted into the mapping expression, and the calibration values ​​of the parameters to be calibrated can be obtained by solving methods such as least squares, thus obtaining the known mapping relationship of the parameters to be calibrated.

[0165] In one possible implementation, the mapping relationship includes: a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle. The corresponding mapping expressions include: a first mapping expression for the first mapping relationship and a second mapping expression for the second mapping relationship. The sampled angle values ​​of the PTZ camera include: sampled values ​​for the pitch angle and sampled values ​​for the horizontal rotation angle. The implementation of S23 above can include the following S231 to S232:

[0166] S231: Substitute the pitch angle sample value and pixel coordinate sample value from each set of sampled data into the first mapping expression, solve for the calibration value of the parameter to be calibrated in the first mapping expression, and obtain the first mapping relationship.

[0167] Specifically, the first mapping expression mentioned above takes the form of the following formula (7): α≈a t x 2 +b t y 2 +c t xy+dt x+e t y+f t Formula (7)

[0168] In this step, each group of sampled data (pan) can be... c ,tilt c x c y c tilt in ) c x c y c Substituting into the above formula (7), let α = tilt c , x = x c y = y c By using tilt in multiple sets of sampled data c x c y c Substituting each equation into formula (7) yields multiple equations, each with an unknown variable a. t b t c t d t e t f t Based on these multiple equations, a system of equations is constructed, and the system of equations is solved using methods such as least squares to obtain a. t b t c t d t e t f t The value of a, and the value of a t b t c t d t e t f t The value of is used as the calibration value of the parameter to be calibrated in the first mapping expression above, and finally the first mapping relationship is obtained.

[0169] It is understandable that, for this first mapping relationship, a t b t c t d t e t f t Let be a known quantity, and α, x, and y be unknown quantities. In S121 above, the target pixel coordinates (x, y) are... i y i Substitute into the first mapping relation, that is, in the first mapping relation, let x = x i y = y i Thus, the target parameter value of the pitch angle, tilt, can be calculated. i tilti =α≈a t x i 2 +b t y i 2 +c t x i y i +d t x i +e t y i +f t

[0170] S232: Substitute the sampled values ​​of the horizontal rotation angle and pixel coordinates from each set of sampled data into the second mapping expression, solve for the calibration values ​​of the parameters to be calibrated in the second mapping expression, and obtain the second mapping relationship.

[0171] Specifically, the second mapping expression mentioned above takes the form of the following formula (8): β≈a p x 2 +b p y 2 +c p xy+d p x+e p y+f p Formula (8)

[0172] In this step, each group of sampled data (pan) can be... c ,tilt c x c y c pan in ) c x c y c Substituting into the above formula (8), let β = pan c , x = x c y = y c By using pan from multiple sets of sampled data c x c y c Substituting each equation into formula (8) above, we can obtain multiple equations, each with an unknown variable of a. p b p c p d p e p f p Based on these multiple equations, a system of equations is constructed, and the system of equations is solved using methods such as least squares to obtain a. t b t c t d t e tf t The value of a, and the value of a t b t c t d t e t f t The value of is used as the calibration value of the parameter to be calibrated in the second mapping expression above, and finally the second mapping relationship is obtained.

[0173] It is understandable that, for this second mapping relationship, a t b t c t d t e t f t β, x, and y are known quantities, while β, x, and y are unknown quantities. In S122 above, the target pixel coordinates (x, y) are... i y i Substitute into the second mapping relation, that is, in the second mapping relation, let x = x i y = y i Thus, the target parameter value pan of the horizontal rotation angle can be calculated. i pan i =β≈a p x i 2 +b p y i 2 +c p x i y i +d p x i +e p y i +f p

[0174] In this embodiment, after obtaining the first and second mapping relationships, during the remote conference, it is only necessary to obtain the target pixel coordinates of the object to be zoomed in on in the camera's image. Typically, the pixel coordinates of the center point of the object to be zoomed in on in the camera's image are used as the target pixel coordinates. Then, by substituting these target pixel coordinates into the first and second mapping relationships, the target parameter values ​​of the pitch angle and the horizontal rotation angle can be directly obtained. Based on these target parameter values, it is beneficial to control the center movement of the PTZ camera to align with the center of the object to be zoomed in on, so as to output a close-up image for the object to be zoomed in on.

[0175] For example, the control parameters of a PTZ camera also include a zoom parameter, the value of which can be set according to the size of the object to be zoomed in on and the composition ratio of the close-up image. This zoom parameter value can be preset as needed, or it can be adjusted according to user requirements during the output of the close-up image; this embodiment does not impose specific limitations on this.

[0176] In one possible implementation, a schematic diagram of the remote conferencing system is shown in Figure 6, which further includes at least one other display device 103 communicatively connected to the display device 101 shown in Figure 1. The display device 101 obtains a close-up image captured by the PTZ camera 102 and sends the close-up image to the aforementioned at least one other display device 103 for display. A camera is installed at position D on the other display device 103.

[0177] It is understood that there may be multiple participants in a remote conference, therefore, there may also be multiple other display devices 103. This embodiment does not specifically limit the number. By obtaining a close-up image captured by the PTZ camera 102 from the display device 101 and sending it to the aforementioned at least one other display device 103 for display, it is beneficial for multiple participants in the remote conference to be able to see the close-up image captured by the PTZ camera 102. By displaying the close-up image on multiple display devices, it is ensured that all participants can receive important information synchronously, reducing information transmission delays and enhancing the interactivity and immersive experience of the remote conference.

[0178] For example, as shown in Figure 6, a PTZ camera 104 can be set up in the meeting scene where the other display device 103 is located. The PTZ camera 104 is used to capture close-up images of the objects to be captured in the meeting scene where the other display device 103 is located. After the other display device 103 obtains the close-up images captured by the PTZ camera 104, it can send them to the display device 101 for display.

[0179] In some embodiments, the flowchart of the method for capturing close-up shots can be referred to in Figure 7, including:

[0180] Step 601: Securely install the PTZ camera and display device, and keep the relative positions between the PTZ camera and display device fixed.

[0181] Step 602: Model the combined system consisting of the PTZ camera and the display device to obtain the mapping expression of the mapping relationship.

[0182] Specifically, the schematic diagram of the model can be found in Figure 5 above, and the method for determining the mapping expression can be found in the description above. To avoid repetition, it will not be repeated here.

[0183] Step 603: Calibrate the parameters to be calibrated in the mapping expression to obtain the mapping relationship.

[0184] Specifically, as mentioned above, the calibration value of the parameter to be calibrated can be calculated by acquiring multiple sets of sampled data and using methods such as least squares, so as to obtain the mapping relationship.

[0185] Step 604: Determine the target pixel coordinates of the object to be zoomed in on in the camera's image.

[0186] Step 605: Substitute the target pixel coordinates into the above mapping relationship to calculate the target parameter values ​​of the PTZ camera's pitch angle and horizontal rotation angle.

[0187] Step 606: Based on the target parameter values ​​of the pitch angle and the horizontal rotation angle of the PTZ camera, drive the PTZ camera to move toward the object to be zoomed in, so that the PTZ camera outputs a close-up image of the object to be zoomed in.

[0188] In this embodiment, the linkage between the PTZ camera and the camera built into the display device can be flexibly realized, improving the imaging effect of the close-up images output by the PTZ camera. Furthermore, as can be seen from the expressions of the first and second mapping relationships described above, the final derived first and second mapping relationships do not actually contain parameters such as the intrinsic and extrinsic parameters and distortion parameters of the PTZ camera and the camera. Therefore, this embodiment does not require accurate calibration and solution of the calibration values ​​of the intrinsic and extrinsic parameters and distortion parameters of the PTZ camera and the camera, saving the complex process of calibrating these parameters. Meanwhile, this embodiment can determine the angle control parameters of the PTZ camera even before the PTZ camera captures the object to be photographed, without having to calculate the rotation angle required for the PTZ camera to align with the object after the PTZ camera has captured the object. This saves the computing power and time resources required for the PTZ camera to first capture the object and then calculate the rotation angle through feature matching. It is beneficial to obtain the target parameter value of the PTZ camera's angle control parameters quickly and efficiently, and control the PTZ camera to move so that the center of the PTZ camera is precisely aligned with the object to be photographed based on the target parameter value, which helps to improve the output efficiency and accuracy of close-up images.

[0189] Figure 8 is a schematic diagram of the structure of a close-up image output device provided in an embodiment of this application.

[0190] For example, as shown in Figure 8, the device 700 is applied to a remote conferencing system, which includes at least one PTZ camera and a display device. The display device is equipped with a camera. The device 700 includes:

[0191] The acquisition module 701 is used to acquire the image captured by the camera mentioned above;

[0192] Detection module 702 is used to detect the target area where the object to be zoomed in is located in the above-mentioned imaging image;

[0193] The determining module 703 is used to determine the parameter values ​​of the angle control parameters of the PTZ camera based on the target area and the set mapping relationship; wherein, the mapping relationship includes the relationship between the position coordinates of each area in the imaging image of the camera and the angle control parameters of the PTZ camera.

[0194] The control module 704 is used to control the parameter value of the angle control parameter to make the PTZ camera move toward the object to be zoomed in and capture a close-up image of the object to be zoomed in.

[0195] In one possible implementation, the device 700 further includes a receiving module for receiving a close-up view presentation request of the object to be close-up in the imaging image before detecting the target area where the object to be close-up is located in the imaging image.

[0196] In one possible implementation, the remote conferencing system further includes at least one other display device that is communicatively connected to the display device. The device 700 also includes a transmission module for obtaining a close-up image captured by a PTZ camera and transmitting the close-up image to the at least one other display device for display.

[0197] In one possible implementation, the mapping relationship includes: the connection between the pixel coordinates of each region in the image of the camera and the parameter values ​​of the angle control parameters of the PTZ camera; the determining module 703 is specifically used to: determine the target pixel coordinates of the target region in the image of the camera; and substitute the target pixel coordinates into the mapping relationship to obtain the target parameter values ​​of the angle control parameters of the PTZ camera.

[0198] In one possible implementation, the aforementioned angle control parameters include: pitch angle and horizontal rotation angle, and the aforementioned mapping relationship includes a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; the determining module 703 is specifically used to: substitute the aforementioned target pixel coordinates into the aforementioned first mapping relationship to obtain the target parameter value of the pitch angle of the aforementioned PTZ camera; and substitute the aforementioned target pixel coordinates into the aforementioned second mapping relationship to obtain the target parameter value of the horizontal rotation angle of the aforementioned PTZ camera.

[0199] In one possible implementation, the apparatus further includes: a mapping relationship determination module, used to construct a mapping expression for the mapping relationship; wherein the mapping expression includes parameters to be calibrated;

[0200] Multiple sets of sampling data are acquired; each set of the above sampling data includes: when the above PTZ camera is facing a certain direction, the sampled value of the angle of the above PTZ camera and the sampled value of the pixel coordinates corresponding to the above camera. The sampled value of the pixel coordinates refers to: when the above PTZ camera is facing the above direction, the pixel coordinates of a specified pixel in the image of the above PTZ camera are mapped to the mapped coordinates in the image of the above camera; based on the above multiple sets of sampling data and the above mapping expression, the calibration value of the above parameter to be calibrated is solved to obtain the above mapping relationship.

[0201] In one possible implementation, the mapping relationship determination module is specifically used to acquire the image images of the PTZ camera and the camera for each direction the PTZ camera faces, and perform feature point matching on the PTZ camera image images and the camera image images to calculate the homography transformation matrix between the PTZ camera and the camera when the PTZ camera faces the direction; based on the homography transformation matrix between the PTZ camera and the camera when the PTZ camera faces the direction, map the pixel coordinates of a specified pixel in the PTZ camera image image to the camera image image to obtain the mapped coordinates, and use the mapped coordinates as the sampled values ​​of the corresponding pixel coordinates of the camera; use the angle value of the PTZ camera when facing the direction as the sampled value of the angle of the PTZ camera; use the sampled value of the angle of the PTZ camera and the sampled value of the corresponding pixel coordinates of the camera as a set of sampled data when the PTZ camera faces one direction; and obtain multiple sets of sampled data when the PTZ camera faces different directions by changing the direction the PTZ camera faces.

[0202] In one possible implementation, the mapping relationship determination module is specifically used to determine a first relationship expression between the close-up angle of the PTZ camera and the target angle; wherein the target angle refers to the angle between the principal ray of the camera and the target direction, and at the close-up angle, the optical axis of the PTZ camera coincides with the principal ray of the PTZ camera; determine a second relationship expression between the target angle and the pixel coordinates of the pixels in the image of the camera; and construct a mapping expression for the mapping relationship based on the first relationship expression and the second relationship expression.

[0203] In one possible implementation, the close-up angle of the PTZ camera includes a close-up pitch angle and a close-up horizontal rotation angle; the target angle includes a first target angle and a second target angle; the target direction includes a first direction and a second direction; the first direction is the optical axis direction of the camera; the second direction is perpendicular to the first direction; the first target angle is the angle between the main ray of the camera and the first direction; and the second target angle is the angle between the main ray of the camera and the second direction. The mapping relationship determination module is specifically used to determine a first relationship expression between the close-up pitch angle of the PTZ camera and the first target angle, and to determine a first relationship expression between the close-up horizontal rotation angle of the PTZ camera and the second target angle; to determine a second relationship expression between the first target angle and the pixel coordinates of the pixels in the image frame of the camera, and to determine a second relationship expression between the second target angle and the pixel coordinates of the pixels in the image frame of the camera.

[0204] In one possible implementation, the mapping relationship includes: a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; the mapping expression includes: a first mapping expression for the first mapping relationship and a second mapping expression for the second mapping relationship; the sampled angle values ​​of the PTZ camera include: sampled values ​​of the pitch angle and sampled values ​​of the horizontal rotation angle; the mapping relationship determination module is specifically used to substitute the sampled values ​​of the pitch angle and the sampled values ​​of the pixel coordinates in each set of the above sampled data into the first mapping expression, solve for the calibration values ​​of the parameters to be calibrated in the first mapping expression, and obtain the first mapping relationship; and to substitute the sampled values ​​of the horizontal rotation angle and the sampled values ​​of the pixel coordinates in each set of the above sampled data into the second mapping expression, solve for the calibration values ​​of the parameters to be calibrated in the second mapping expression, and obtain the second mapping relationship.

[0205] Figure 9 is a schematic diagram of the structure of a display device provided in an embodiment of this application.

[0206] For example, as shown in FIG9, the display device 800 includes a memory 801 and a processor 802, wherein the memory 801 stores executable program code 8011, and the processor 802 is used to call and execute the executable program code 8011 to perform a method for capturing a close-up image.

[0207] Furthermore, this application also protects an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a close-up image capture method provided in this application.

[0208] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0209] When each functional module is divided according to its corresponding function, the device may further include an acquisition module, a detection module, a determination module, and a control module. It should be noted that all relevant content regarding the steps involved in the above method embodiments can be referenced to the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0210] It should be understood that the device provided in this embodiment is used to perform the above-described method for capturing close-up images, and therefore can achieve the same effect as the above-described implementation method.

[0211] When using integrated units, the device may include a processing module and a storage module. When applied to a display device, the processing module can be used to control and manage the operation of the display device. The storage module can be used to support the display device in executing relevant program code.

[0212] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0213] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a close-up image shooting method provided in the above embodiments.

[0214] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described method steps to achieve the close-up image capture method provided in the above embodiment.

[0215] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to achieve a close-up image capture method provided in the above embodiment.

[0216] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0217] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0218] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0219] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for shooting close-up shots, characterized in that, Applied to a remote conferencing system, the remote conferencing system including at least one PTZ camera and a display device, the display device being equipped with a camera, the method comprising: Acquire the image captured by the camera; Detect the target area where the object to be zoomed in is located in the imaging image; Based on the target area and the established mapping relationship, the target parameter value of the angle control parameter of the PTZ camera is determined; wherein, the mapping relationship includes: the relationship between each area in the image of the camera and the parameter value of the angle control parameter of the PTZ camera; Based on the target parameter value of the angle control parameters, the PTZ camera moves toward the object to be zoomed in, and captures a close-up image of the object.

2. The method according to claim 1, characterized in that, Before detecting the target area where the object to be zoomed in is located in the imaging image, the method further includes: Receive the requirement to display a close-up image of the object to be shown in close-up.

3. The method according to claim 2, characterized in that, The remote conferencing system further includes: at least one other display device communicatively connected to the display device, and the method further includes: Obtain a close-up image captured by the PTZ camera and send the close-up image to the at least one other display device for display.

4. The method according to claim 1, characterized in that, The mapping relationship includes the connection between the pixel coordinates of each region in the image captured by the camera and the parameter values ​​of the angle control parameters of the PTZ camera. The step of determining the target parameter value of the PTZ camera's angle control parameters based on the target area and the set mapping relationship includes: Determine the target pixel coordinates of the target area in the image captured by the camera; By substituting the target pixel coordinates into the mapping relationship, the target parameter values ​​of the angle control parameters of the PTZ camera are obtained.

5. The method according to claim 4, characterized in that, The angle control parameters include: pitch angle and horizontal rotation angle, and the mapping relationship includes a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; The step of mapping the target pixel coordinates into the mapping relationship to obtain the parameter values ​​of the angle control parameters of the PTZ camera includes: The target pixel coordinates are substituted into the first mapping relationship to obtain the target parameter value of the pitch angle of the PTZ camera; By substituting the target pixel coordinates into the second mapping relationship, the target parameter value of the horizontal rotation angle of the PTZ camera is obtained.

6. The method according to claim 1, characterized in that, The mapping relationship is determined in the following way: Construct a mapping expression for the mapping relationship; wherein the mapping expression includes parameters to be calibrated; Acquire multiple sets of sampling data; wherein, each set of sampling data includes: when the PTZ camera is facing a direction, the sampling value of the angle of the PTZ camera and the sampling value of the pixel coordinates corresponding to the camera, wherein the sampling value of the pixel coordinates refers to: when the PTZ camera is facing the direction, the pixel coordinates of a specified pixel in the imaging image of the PTZ camera are mapped to the mapping coordinates in the imaging image of the camera; Based on the multiple sets of sampled data and the mapping expression, the calibration value of the parameter to be calibrated is solved to obtain the mapping relationship.

7. The method according to claim 6, characterized in that, The acquisition of multiple sets of sampling data includes: When the PTZ camera faces a certain direction, the imaging images of the PTZ camera and the camera are acquired, and feature point matching is performed on the imaging images of the PTZ camera and the camera to calculate the homography transformation matrix between the PTZ camera and the camera when the PTZ camera faces the direction. Based on the homography transformation matrix between the PTZ camera and the camera when the PTZ camera is facing the direction, the pixel coordinates of a specified pixel in the image of the PTZ camera are mapped to the image of the camera to obtain the mapped coordinates, and the mapped coordinates are used as the sampled values ​​of the corresponding pixel coordinates of the camera. The angle value of the PTZ camera when it faces the direction is used as the sampled value of the angle of the PTZ camera; The sampled values ​​of the angle of the PTZ camera and the sampled values ​​of the pixel coordinates corresponding to the camera are used as a set of sampled data when the PTZ camera is facing a certain direction; By changing the orientation of the PTZ camera, multiple sets of sampling data can be obtained when the PTZ camera is facing different directions.

8. The method according to claim 6, characterized in that, The mapping expression for constructing the mapping relationship includes: A first relationship expression is determined between the close-up angle of the PTZ camera and the target angle; wherein, the target angle refers to the angle between the principal ray of the camera and the target direction, and at the close-up angle, the PTZ camera... The optical axis of the machine coincides with the principal ray of the PTZ camera; Determine a second relationship expression between the target angle and the pixel coordinates of the pixels in the image captured by the camera; Based on the first relational expression and the second relational expression, construct the mapping expression for the mapping relationship.

9. The method according to claim 8, characterized in that, The close-up angles of the PTZ camera include: close-up pitch angle and close-up horizontal rotation angle; the target angles include: a first target angle and a second target angle; the target directions include a first direction and a second direction, the first direction being the optical axis direction of the camera, the second direction being perpendicular to the first direction, the first target angle being the angle between the main ray of the camera and the first direction, and the second target angle being the angle between the main ray of the camera and the second direction. The first relationship expression for determining the close-up angle of the PTZ camera and the angle between the target includes: Determine a first relationship expression between the close-up pitch angle of the PTZ camera and the angle between the first target, and determine a first relationship expression between the close-up horizontal rotation angle of the PTZ camera and the angle between the second target; The second relationship expression for determining the target angle and the pixel coordinates of the pixels in the image captured by the camera includes: A second relational expression is determined between the first target angle and the pixel coordinates of the pixels in the image of the camera, and a second relational expression is determined between the second target angle and the pixel coordinates of the pixels in the image of the camera.

10. The method according to claim 6, characterized in that, The mapping relationship includes: a first mapping relationship corresponding to the pitch angle and a second mapping relationship corresponding to the horizontal rotation angle; the mapping expression includes: a first mapping expression for the first mapping relationship and a second mapping expression for the second mapping relationship; the sampled angle values ​​of the PTZ camera include: the sampled pitch angle values ​​and the sampled horizontal rotation angle values. The step of solving for the calibration value of the parameter to be calibrated based on the multiple sets of sampled data and the mapping expression to obtain the mapping relationship includes: Substitute the sampled values ​​of the pitch angle and the pixel coordinates from each set of sampled data into the first mapping expression, solve for the calibration values ​​of the parameters to be calibrated in the first mapping expression, and obtain the first mapping relationship; Substitute the sampled values ​​of the horizontal rotation angle and the pixel coordinates from each set of sampled data into the second mapping expression, solve for the calibration values ​​of the parameters to be calibrated in the second mapping expression, and obtain the second mapping relationship.

11. A device for capturing close-up shots, characterized in that, Applied to a remote conferencing system, the remote conferencing system includes at least one PTZ camera and a display device, the display device being equipped with a camera, the device comprising: The acquisition module is used to acquire the image captured by the camera; The detection module is used to detect the target area where the object to be zoomed in is located in the imaging image; The determining module is used to determine the parameter values ​​of the angle control parameters of the PTZ camera based on the target area and the set mapping relationship; wherein, the mapping relationship includes: the relationship between each area in the image of the camera and the angle control parameters of the PTZ camera; The control module is used to control the angle parameters to make the PTZ camera move toward the object to be zoomed in, and capture a close-up image of the object to be zoomed in.

12. A display device, characterized in that, The display device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the display device to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 10.

Images