Static gesture interaction method, PTZ camera and computer storage medium

Abstract

The invention relates to the technical field of security and protection, and discloses a static gesture interaction method, a PTZ camera and a computer storage medium. The static gesture interaction method is applied to a PTZ camera, and comprises the following steps: collecting a video stream of a meeting place in real time, and identifying human body key point information of a plurality of participants from the video stream; determining gesture categories of the plurality of participants according to the human body key point information of the plurality of participants; according to the gesture types of the plurality of participants, identifying a speaker, and carrying out cross-device or cross-time tracking on the speaker; determining human body position information of the speaker according to the identified speaker; and according to the human body position information of the speaker, focus tracking lens parameters of a focus tracking lens of the PTZ camera are automatically adjusted, and zooming operation of the PTZ camera is automatically realized. Therefore, static gesture interaction based on the PTZ camera is achieved, zooming operation of the PTZ camera is automatically achieved, the operation mode is natural, visual and convenient, and user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of security technology, and in particular to a static gesture interaction method, a PTZ camera and a computer storage medium. Background Art

[0002] In the security field, PTZ (short for Pan / Tilt / Zoom) cameras have become one of the essential monitoring devices for security monitoring systems because they can perform autonomous and automatic PTZ tracking.

[0003] At present, the operation of traditional PTZ cameras mainly relies on input devices such as remote controls, keyboards or mice. Users need to control the movement, zoom and other functions of the PTZ camera by pressing buttons or clicking the mouse. In addition, traditional PTZ cameras do not have automatic focus tracking capabilities and need to be manually focused by remote controls. For example, in a multi-person meeting, there are several employees in front of the PTZ camera. During a remote meeting with customers, all five employees can raise their hands to speak. If employee A raises his hand to speak, the PTZ camera's focus tracking lens will cut the meeting screen to the speaker, employee A, so that customers can watch employee A's speech process. To meet such interactive needs, traditional PTZ cameras need to be accompanied by someone who specializes in operating the focus tracking lens zoom. This manual method is cumbersome, not intuitive and convenient, and slow to respond, affecting the efficiency of interaction. Summary of the invention

[0004] The embodiments of the present invention aim to provide a static gesture interaction method, a PTZ camera and a computer storage medium, which can solve the problem that the existing PTZ camera adopts cumbersome manual operation, is not intuitive and convenient, and has slow response, which affects the interaction efficiency.

[0005] In order to solve the above technical problems, a first aspect of the present invention provides a static gesture interaction method, which is applied to a PTZ camera. The static gesture interaction method includes:

[0006] Collect the video stream of the conference site in real time, and identify the key points of the human bodies of several participants from the video stream;

[0007] Determine the gesture categories of the participants according to the key point information of the participants;

[0008] Identify the speaker based on the gesture categories of several participants, and track the speaker across devices or across time;

[0009] Determining body position information of the speaker according to the identified speaker;

[0010] According to the body position information of the speaker, the focus lens parameters of the focus lens of the PTZ camera are automatically adjusted to automatically implement the zoom operation of the PTZ camera.

[0011] Optionally, the real-time acquisition of the video stream of the conference site and identification of key body point information of several conference participants from the video stream include:

[0012] Get the video stream of the venue from the wide-angle lens of the PTZ camera;

[0013] Performing human body detection on the video stream to obtain human body range frames of several participants;

[0014] According to the human body range frames of several participants, the human body key point information of several participants is identified through a preset algorithm.

[0015] Optionally, determining the gesture categories of the plurality of participants according to the body key point information of the plurality of participants includes:

[0016] According to the human body key point information of several participants, the hand positions of several participants are predicted, and the hand range frames of several participants are output;

[0017] According to the hand range frames of the participants, the key point information of the gestures of the participants is obtained;

[0018] According to the gesture key point information of several participants, gesture categories corresponding to the several participants are obtained.

[0019] Optionally, identifying a speaker according to the gesture categories of the plurality of participants and tracking the speaker across devices or across time includes:

[0020] It is determined whether the gesture categories of several participants are subsets of a preset trigger static gesture library. If it is determined that the gesture category of one of the participants is a subset of the preset trigger static gesture library, the participant is identified as a speaker, a human body range frame corresponding to the participant is found, the mark is set as the speaker's human body range frame, and tracking zoom is triggered to track the speaker across devices and / or across time.

[0021] Optionally, determining the body position information of the speaker according to the identified speaker includes:

[0022] After triggering the tracking of the speaker, determining whether the speaker's gesture is a triggering gesture through gesture recognition based on the key point information of the speaker's body;

[0023] If it is determined that the speaker's gesture is a trigger gesture, further determining whether the trigger gesture is a focus locking gesture;

[0024] If it is determined that the trigger gesture is a focus locking gesture, the body position information of the speaker is determined according to the body key point information of the speaker.

[0025] Optionally, automatically adjusting the focus lens parameters of the focus lens of the PTZ camera according to the body position information of the speaker to automatically implement the zoom operation of the PTZ camera includes:

[0026] Determining intrinsic parameters and extrinsic parameters of a virtual camera using a camera model according to the body position information of the speaker;

[0027] Converting the internal and external parameters of the virtual camera into focus lens parameters of the PTZ camera;

[0028] According to the focus tracking lens parameters of the PTZ camera, the focal length of the focus tracking lens and the pan / tilt rotation angle of the PTZ camera are automatically adjusted to automatically implement the zoom operation of the PTZ camera.

[0029] Optionally, the static gesture interaction method further includes: restoring the PTZ camera to a default position and stopping adjustment of the focal length according to an unlocking gesture of the speaker.

[0030] Accordingly, the second embodiment of the present invention provides a PTZ camera, which is applied to the static gesture interaction method described in the first embodiment of the present invention, wherein the PTZ camera includes a wide-angle lens, a human body detection module, a speaker tracking module, a human body key point detection module, a hand extension module, a gesture key point detection module, a gesture differentiation module, a human body position calculation module, a focus tracking calculation module and a focus tracking lens; wherein:

[0031] The wide-angle lens is arranged at the bottom of the PTZ camera and is used to perform real-time panoramic detection of the venue, obtain the video stream of the venue, and detect the participants in the venue in real time;

[0032] The human body detection module is used to perform human body detection on the video stream to obtain human body range frames of several participants;

[0033] The speaker tracking module is used to track the speaker across devices or across time after identifying the speaker among a number of conference participants;

[0034] The human body key point detection module is used to identify the human body key point information of several participants according to the human body range frames of several participants through a preset algorithm;

[0035] The hand expansion module is used to predict the hand positions of several participants according to the human body key point information or the hand range frame information of the historical frames, and output the hand range frames of several participants;

[0036] The gesture key point detection module is used to obtain the gesture key point information of several participants according to the hand range frames of several participants;

[0037] The gesture distinguishing module is used to obtain gesture categories corresponding to a number of participants according to the gesture key point information of a number of participants;

[0038] The human body position calculation module is used to perform geometric calculations based on the human body key point information of the speaker and the parameters of the wide-angle lens, obtain the posture information of the speaker relative to the wide-angle lens of the PTZ camera by substituting into a standard human body parameter model, and determine the human body position information of the speaker;

[0039] The focus tracking calculation module is used to determine the intrinsic parameters and extrinsic parameters of the virtual camera using the camera model according to the body position information of the speaker, and convert the intrinsic and extrinsic parameters of the virtual camera into focus tracking lens parameters of the PTZ camera;

[0040] The tracking lens is arranged on the top of the PTZ camera, and is used to automatically adjust the focal length and the pan / tilt rotation angle according to the tracking lens parameters of the PTZ camera, so as to automatically realize the zoom operation of the PTZ camera.

[0041] Accordingly, a third aspect of the present invention provides a PTZ camera, comprising: a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the static gesture interaction method described in the first aspect of the present invention.

[0042] Correspondingly, the fourth aspect embodiment of the present invention provides a computer storage medium, on which a program of a static gesture interaction method is stored. When the program of the static gesture interaction method is executed by a processor, the static gesture interaction method described in the first aspect embodiment of the present invention is implemented.

[0043] Compared with the prior art, the present invention provides a static gesture interaction method, a PTZ camera and a computer storage medium. The static gesture interaction method is applied to a PTZ camera, including: real-time acquisition of the video stream of the venue, identifying the key point information of the human body of several participants from the video stream; determining the gesture categories of several participants according to the key point information of the human body of several participants; identifying the speaker according to the gesture category of several participants, and tracking the speaker across devices or across time; determining the human body position information of the speaker according to the identified speaker; automatically adjusting the focus lens parameters of the focus lens of the PTZ camera according to the human body position information of the speaker, and automatically realizing the zoom operation of the PTZ camera. Thus, the key point information and gesture categories of several participants are identified from the video stream through the human body key point recognition technology; the speaker is identified from the gesture categories of several participants through the gesture recognition technology, and accurate speaker recognition and tracking are realized; and the speaker's gesture is recognized through the gesture recognition technology, and the zoom function of the PTZ camera is automatically triggered, and the focal length is automatically adjusted. Thus, the PTZ camera combines the human key point recognition and gesture recognition technology, and enables the PTZ camera to have gesture interaction function and automatic focus tracking capability, realizes static gesture interaction based on the PTZ camera, and automatically realizes the zoom operation of the PTZ camera, without the need for special personnel to manually operate the focus tracking lens zoom. The operation method is natural, intuitive, convenient, and responsive, which improves the interaction efficiency and user experience. This can solve the problem that the existing PTZ camera adopts cumbersome manual operation, is not intuitive and convenient, and has slow response, which affects the interaction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0045] Figure 1 It is a flowchart of a static gesture interaction method provided by the present invention;

[0046] Figure 2 is a flow chart of step S1 in a static gesture interaction method provided by the present invention;

[0047] Figure 3 It is a schematic diagram of representing key points of a human body as 17 joints in a static gesture interaction method provided by the present invention;

[0048] Figure 4 is a flow chart of step S2 in a static gesture interaction method provided by the present invention;

[0049] Figure 5It is a schematic diagram of defining 21 key points of hand gestures in a static gesture interaction method provided by the present invention;

[0050] Figure 6 is a flow chart of step S4 in a static gesture interaction method provided by the present invention;

[0051] Figure 7 is a flow chart of step S5 in a static gesture interaction method provided by the present invention;

[0052] Figure 8 is another flow chart of a static gesture interaction method provided by the present invention;

[0053] Fig. 9 It is a structural schematic diagram of a PTZ camera provided by the present invention;

[0054] Fig.10 It is another structural schematic diagram of a PTZ camera provided by the present invention. DETAILED DESCRIPTION

[0055] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0056] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0057] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0058] PTZ (abbreviation of Pan / Tilt / Zoom) camera, which stands for omnidirectional (left / right / up / down) movement of the PTZ and lens zoom control, can be generally understood as PTZ control. Within the scene monitored by the PTZ camera, when a moving target appears, the user can manually lock (for example, by clicking the mouse to lock the target) or automatically trigger the preset position to lock a moving target, triggering the PTZ camera to perform autonomous and automatic PTZ tracking, and automatically control the PTZ camera's PTZ to rotate in all directions, and perform visually guided automatic tracking of the locked moving target to ensure that the tracked target continues to appear in the center of the lens. This makes the PTZ camera one of the necessary monitoring equipment for security monitoring systems.

[0059] At present, the operation of traditional PTZ cameras mainly relies on input devices such as remote controls, keyboards or mice. Users need to control the movement, zoom and other functions of the PTZ camera by pressing buttons or clicking the mouse. In addition, traditional PTZ cameras do not have gesture interaction functions and automatic focus tracking capabilities, and need to be manually focused through a remote control. For example, in a multi-person meeting, there are several employees in front of the PTZ camera, and they are having a remote meeting with customers. These five employees can raise their hands to speak. If employee A raises his hand to speak, the focus tracking lens of the PTZ camera will cut the meeting screen to the speaker employee A, so that customers can watch the speech process of employee A. To meet such interactive needs, traditional PTZ cameras need to be accompanied by someone who specializes in operating the focus tracking lens zoom. This manual method is cumbersome, not intuitive and convenient, and slow to respond, which affects the efficiency of interaction.

[0060] To this end, the present invention provides a static gesture interaction method, which is applied to a PTZ camera. By real-time acquisition of the video stream of the venue, the human body key point information and gesture categories of several participants are identified from the video stream through the human body key point recognition technology; the speaker is identified from the gesture categories of several participants through the gesture recognition technology to achieve accurate speaker identification and tracking; and the speaker's gesture is then identified through the gesture recognition technology to automatically trigger the zoom function of the PTZ camera and automatically adjust the focal length. Thus, the PTZ camera combines the human body key point recognition and gesture recognition technology, so that the PTZ camera has the gesture interaction function and automatic focus tracking capability, realizes the static gesture interaction based on the PTZ camera, and automatically realizes the zoom operation of the PTZ camera, without the need for a dedicated person to manually operate the focus tracking lens zoom, the operation method is natural, intuitive, convenient, and responsive, which improves the interaction efficiency and user experience. Thus, the problem that the existing PTZ camera adopts cumbersome manual operation, is not intuitive and convenient, and has a slow response, which affects the interaction efficiency can be solved.

[0061] In order to facilitate understanding of the above inventive concept of the present invention, the above inventive concept of the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments.

[0062] In one embodiment, Figure 1 As shown, the present invention provides a static gesture interaction method, which is applied to a PTZ camera. The static gesture interaction method includes:

[0063] S1, collect the video stream of the conference site in real time, and identify the key points of the human body of several participants from the video stream;

[0064] S2. determining the gesture categories of the participants according to the key point information of the participants;

[0065] S3, identifying the speaker based on the gesture categories of several participants, and tracking the speaker across devices or across time;

[0066] S4, determining the body position information of the speaker according to the identified speaker;

[0067] S5. Automatically adjust the focus lens parameters of the focus lens of the PTZ camera according to the body position information of the speaker, and automatically implement the zoom operation of the PTZ camera.

[0068] In this embodiment, the human body key point information of several participants is identified from the video stream by real-time acquisition of the video stream of the venue; the gesture categories of several participants are determined based on the human body key point information of several participants; the speaker is identified based on the gesture categories of several participants, and the speaker is tracked across devices or across time; the human body position information of the speaker is determined based on the identified speaker; the focus lens parameters of the focus lens of the PTZ camera are automatically adjusted based on the human body position information of the speaker, and the zoom operation of the PTZ camera is automatically realized. Thus, the human body key point information and gesture categories of several participants are identified from the video stream through the human body key point recognition technology; the speaker is identified from the gesture categories of several participants through the gesture recognition technology, and accurate speaker recognition and tracking are realized; and the speaker's gesture is recognized through the gesture recognition technology, and the zoom function of the PTZ camera is automatically triggered, and the focal length is automatically adjusted. Thus, the PTZ camera combines the human key point recognition and gesture recognition technology, and enables the PTZ camera to have gesture interaction function and automatic focus tracking capability, realizes static gesture interaction based on the PTZ camera, and automatically realizes the zoom operation of the PTZ camera, without the need for special personnel to manually operate the focus tracking lens zoom. The operation method is natural, intuitive, convenient, and responsive, which improves the interaction efficiency and user experience. This can solve the problem that the existing PTZ camera adopts cumbersome manual operation, is not intuitive and convenient, and has slow response, which affects the interaction efficiency.

[0069] In one embodiment, Figure 2As shown, in step S1, the video stream of the conference site is collected in real time, and the key point information of the human bodies of several participants is identified from the video stream; including:

[0070] S11. Obtain the video stream of the conference site from the wide-angle lens of the PTZ camera.

[0071] The PTZ camera includes a wide-angle lens and a tracking lens, wherein the wide-angle lens is arranged at the bottom of the PTZ camera and is used for real-time panoramic detection of the venue, obtaining the video stream of the venue, and real-time detection of participants in the venue.

[0072] The tracking lens is arranged on the top of the PTZ camera, and is used to automatically adjust the focal length and the pan / tilt rotation angle according to the tracking lens parameters of the PTZ camera, automatically realize the zoom operation of the PTZ camera, and accurately focus on the focal length required by the speaker.

[0073] S12: Perform human body detection in the video stream to obtain human body range frames of several participants.

[0074] Specifically, human body detection is performed in the video stream through a computer vision algorithm, and N human body range frames are generated for the detected participants in the form of pixel coordinates under the photo (there are N participants in the video stream), and human body range frames of several participants are obtained. Each participant has a specified human body range frame (x, y, h, w), where x is the horizontal coordinate, y is the vertical coordinate, h is the height, and w is the width.

[0075] More specifically, human body detection is performed on the current frame of the video stream through a computer vision algorithm. After the human body is detected, the current frame of the video stream is cropped to obtain a human body image, and a human body range frame (x, y, h, w) is generated for the human body image in the form of pixel coordinates under the photo. If N participants are detected in the video stream, N human body range frames (x, y, h, w) are generated, thereby obtaining the human body range frames of several participants, wherein x is the horizontal coordinate, y is the vertical coordinate, h is the height, and w is the width.

[0076] S13. According to the body range frames of the several participants, the body key point information of the several participants is identified by a preset algorithm.

[0077] Specifically, the preset algorithm includes a deep learning algorithm, which uses the deep learning algorithm to identify the human body range frames of several participants, obtain the human body key point information of several participants on the image plane of the current frame corresponding to the video stream (if there are N participants in the video stream, there are N human body key point information), and realize accurate identification of the human body key point information of the participants.

[0078] like Figure 3As shown in FIG. 1 , the key points of the human body include 17 joints, namely nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles. The key point information of the human body includes the above 17 joint information.

[0079] In this embodiment, human body detection is performed on the current frame of the video stream through a computer vision algorithm, so that the participants in the video stream can be accurately detected, and the human body key point information of several participants can be accurately identified through a deep learning algorithm.

[0080] In one embodiment, Figure 4 As shown, in step S2, the gesture categories of the participants are determined according to the key point information of the participants; including:

[0081] S21. Predicting the hand positions of the participants based on the key point information of the participants, and outputting the hand range frames of the participants.

[0082] Specifically, for each participant's body key point information, the hand range box (x_1, y_1, h_1, w_1) of each participant is determined (calculated) according to the preset hand position prediction logic, where x_1 is the horizontal coordinate, y_1 is the vertical coordinate, h_1 is the height, and w_1 is the width.

[0083] Among them, the preset hand position prediction logic includes:

[0084] A. Through the hand range frame of the historical frame, extract the movement speed, movement trajectory and other information of the hand in the historical frame, predict the current frame position through the hand range frame of the previous 5 frames, and obtain the first predicted hand range frame.

[0085] B. According to the human body key point information of the participants, including the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, left and right ankles and other human body key points, among which the hand center distance is taken on the extension line of the "left elbow-left arm" / "right elbow-right arm" key points, and the circle of the hand diameter is taken, and its circumscribed rectangle is used as the second predicted hand range frame. The first predicted hand range frame and the second predicted hand range frame are positionally fused. When the IOU (Intersection over Union) value of the first predicted hand range frame and the second predicted hand range frame is large, the overlapping area of ​​the two is taken for the second external expansion ratio expansion and then output as the result of the hand range frame of the current frame. Among them, IOU (Intersection over Union) is used to describe the overlap degree of two frames. The larger the overlapping area, the larger the IOU value.

[0086] C. When the IOU value between the first predicted hand range box and the second predicted hand range box is less than the threshold, if the credibility of the human key point information is lower than the human credibility threshold, the first predicted hand range box is used to perform a second outward expansion ratio expansion and is output as the result of the hand range box of the current frame; otherwise, the second predicted hand range box is used to perform a second outward expansion ratio expansion and is output as the result of the hand range box of the current frame.

[0087] S22. Obtaining gesture key point information of the plurality of conference participants according to the hand range frames of the plurality of conference participants.

[0088] After obtaining the hand range frames of several participants, the gesture key point information of each participant can be obtained according to the hand range frames.

[0089] like Figure 5 As shown in the figure, 21 key points of hand gesture are defined, which describe the wrist, finger end joints, finger tip joints, finger proximal joints, and finger distal joints respectively.

[0090] By detecting the above-mentioned gesture key points at specific positions of the palm and fingers, the position and direction information of the palm and fingers can be obtained, and the positions of the palm and fingers can be located.

[0091] S23. Obtain gesture categories corresponding to the plurality of participants according to the gesture key point information of the plurality of participants.

[0092] Specifically, according to the gesture key point information of several participants, the state of the finger joints is calculated (for example: straight, curled, relaxed), the palm direction is calculated, and the current gesture is determined by combining multiple finger states to determine whether it is included in the preset trigger static gesture library, and the gesture classification and palm direction results are returned, so as to obtain the gesture categories corresponding to several participants by identifying the gesture key point information. Among them, the preset trigger static gesture library includes at least the following static gestures: five fingers open, thumbs up, ok, scissors hand, fist.

[0093] In this embodiment, accurate recognition of the speaker's gestures can be achieved through human body key point recognition and deep learning algorithms, thereby improving the accuracy and robustness of gesture interaction.

[0094] In one embodiment, in step S3, the speaker is identified according to the gesture categories of several participants, and the speaker is tracked across devices or across time; including:

[0095] It is determined whether the gesture categories of several participants are subsets of the preset trigger static gesture library. If it is determined that the gesture category of one of the participants is a subset of the preset trigger static gesture library, the participant is identified as the speaker, the human body range frame corresponding to the participant is found, the mark is set as the speaker's human body range frame, and tracking zoom is triggered to track the speaker across devices and / or across time; otherwise, tracking zoom is not triggered, and the panoramic detection of the wide-angle lens is repeated. In this way, accurate speaker identification and tracking are achieved.

[0096] When tracking zoom is triggered, the wide-angle lens automatic tracking function of the PTZ camera and the automatic zoom function of the focus lens will be triggered. The wide-angle lens of the PTZ camera will automatically track the participants who have triggered the zoom gesture or have the focus locked, and track the speaker when the speaker reappears after walking out of the wide-angle lens. If the speaker does not walk out of the wide-angle lens in a certain frame sequence, the human body range frame of the previous frame is expanded by the first external expansion ratio as the output result of the human body range frame of the current frame to reduce the computing power occupied by the algorithm.

[0097] In one embodiment, Figure 6 As shown, in step S4, the human body position information of the speaker is determined according to the identified speaker; including:

[0098] S41 , after triggering the tracking of the speaker, determining whether the speaker's gesture is a triggering gesture through gesture recognition based on the key point information of the speaker's body.

[0099] Specifically, it is determined whether the speaker's gesture category is a subset of a preset trigger static gesture library. If it is determined that the speaker's gesture category is a subset of the preset trigger static gesture library, the speaker's gesture is determined to be a trigger gesture.

[0100] S42: If it is determined that the speaker's gesture is a trigger gesture, further determine whether the trigger gesture is a focus locking gesture.

[0101] Specifically, it is determined whether the trigger gesture of the speaker is a subset of the preset trigger static gesture library. If it is determined that the trigger gesture of the speaker is a subset of the preset trigger static gesture library, it is determined that the gesture of the speaker is a focus lock gesture. For example, when the speaker's gesture is "five fingers open", it represents a focus lock gesture.

[0102] S43: If it is determined that the trigger gesture is a focus locking gesture, the body position information of the speaker is determined according to the body key point information of the speaker.

[0103] Specifically, the PTZ camera performs geometric calculations based on the speaker's body key point information and the parameters of the wide-angle lens, and obtains the speaker's posture information relative to the wide-angle lens of the PTZ camera by substituting the standard body parameter model, thereby determining the speaker's body position information.

[0104] In this embodiment, through human body key point recognition and deep learning algorithms, accurate recognition of the speaker's gestures can be achieved, improving the accuracy and robustness of gesture interaction; and through gesture interaction, the speaker can control the zoom of the PTZ camera with simple gestures, making the operation more intuitive and natural.

[0105] In one embodiment, Figure 7 As shown, in step S5, the focusing lens parameters of the focusing lens of the PTZ camera are automatically adjusted according to the body position information of the speaker, and the zoom operation of the PTZ camera is automatically realized; including:

[0106] S51 . Determine intrinsic parameters and extrinsic parameters of a virtual camera using a camera model according to the speaker's body position information.

[0107] Specifically, according to the speaker's body position information, the camera model is used to calculate the intrinsic and extrinsic parameters of the virtual camera, including focal length (intrinsic parameters), pan-tilt movement parameters (extrinsic parameters), etc. The camera model can be modeled according to the actual scene and camera parameters to achieve accurate calculation of focal length and pan-tilt movement. Among them, the camera extrinsic parameters define the camera pose (e.g., position, direction, and movement), and the camera intrinsic parameters specify the camera image format (e.g., focal length, pixel size, and image origin).

[0108] The camera model is used to calculate the intrinsic and extrinsic parameters of the virtual camera, including: when the position and size of the object being photographed, as well as the intrinsic and extrinsic parameters of the camera are known, the virtual camera parameters (extrinsic parameters) corresponding to the cropped image (the cropped image is the image obtained by cropping the original image) are calculated. And internal parameters fx, fy, cx, cy), calculate the crop image data through multi-view geometry, generate a data set of "image-camera parameters" type, and use this type of image data to test, train and infer the model with input type "image-camera parameters".

[0109] Virtual camera parameters (external parameters The solution method for the internal parameters fx, fy, cx, cy) is as follows:

[0110] The external parameters of the camera are known, that is, the description of the camera in the world coordinate system:

[0111] The description of the world coordinates in the camera coordinate system is:

[0112] Assume that the 3D coordinates of the captured object are known, for example (gesture key points).

[0113] It is the point cloud position set of gesture points in the world coordinate system.

[0114] The calculation formula for the center position of the target point cloud is as follows:

[0115]

[0116] Calculate the center position of the target point cloud in the camera coordinate system:

[0117]

[0118] Calculate the rotation and translation matrix of the z-axis direction of the camera coordinate system and the center position vector of the target point cloud:

[0119]

[0120] Using the Rodrigues formula, input the z axis (0,0,1) and Find Right now

[0121] Find the positional relationship between the virtual camera and the world coordinate origin, as shown in the following formula:

[0122]

[0123] Calculate the position of the target point cloud in the virtual camera coordinate system:

[0124]

[0125] Given the virtual camera's phase plane size parameters: image w , image h .

[0126] According to the pinhole imaging principle:

[0127]

[0128]

[0129] c x =(image w -1) / 2

[0130] c y =(image h -1) / 2

[0131] Since the pinhole imaging model is used, the distortion coefficient is 0.

[0132] So far, the external parameters of the virtual camera are And the internal parameters fx, fy, cx, cy have been calculated.

[0133] S52: Convert the internal and external parameters of the virtual camera into focus lens parameters of the PTZ camera.

[0134] The image size and focal length of the virtual camera are different from those of the PTZ camera. It is necessary to convert the internal parameters of the virtual camera according to the equivalent conversion principle of the pinhole camera to calculate the equivalent focal length parameters of the PTZ camera's tracking lens.

[0135] Through the speaker's body position information, the vector extending from the origin of the PTZ camera coordinates along the Z axis is the first camera vector, and the speaker's center point to the PTZ camera coordinates is the second camera vector. The focus rotation angle of the PTZ camera coordinates around the X axis / Y axis can be calculated from the first camera vector to the second camera vector.

[0136] The equivalent focal length parameter and the focus rotation angle are combined and output as focus lens parameters, which are provided to the PTZ camera to perform specified focal length transformation and pan / tilt angle control.

[0137] S53. The PTZ camera automatically adjusts the focal length and pan / tilt rotation angle of the tracking lens of the PTZ camera according to the tracking lens parameters of the PTZ camera, automatically implements the zoom operation of the PTZ camera, and accurately focuses on the focal length required by the speaker.

[0138] In this embodiment, by determining the intrinsic and extrinsic parameters of the virtual camera using a camera model based on the speaker's body position information, the focus lens parameters of the PTZ camera, including the focal length and pan / tilt movement parameters, can be accurately calculated; by automatically adjusting the focal length and pan / tilt movement parameters of the PTZ camera's focus lens, precise control of the focus point can be automatically achieved, the focus can be automatically aligned with the focal length required by the speaker, and the zoom operation of the PTZ camera can be automatically achieved, providing better shooting effects.

[0139] In one embodiment, Figure 8 As shown, the static gesture interaction method further includes: S6, according to the speaker's unlocking gesture, restoring the PTZ camera to a default position and stopping the adjustment of the focal length.

[0140] Specifically, after triggering the tracking of the speaker, the gesture of the speaker is determined to be a trigger gesture through gesture recognition based on the key point information of the speaker's body; specifically, it is determined whether the gesture category of the speaker is a subset of a preset trigger static gesture library. If it is determined that the gesture category of the speaker is a subset of the preset trigger static gesture library, then the gesture of the speaker is determined to be a trigger gesture.

[0141] If the speaker's gesture is determined to be a trigger gesture, it is further determined whether the trigger gesture is a focus unlocking gesture. Specifically, it is determined whether the speaker's trigger gesture is a subset of a preset trigger static gesture library. If it is determined that the speaker's trigger gesture is a subset of a preset trigger static gesture library, it is determined that the speaker's gesture is a focus unlocking gesture. For example, when the speaker's gesture is a "clenched fist", it indicates a focus unlocking gesture.

[0142] If the trigger gesture is determined to be an unlock gesture, the zoom function of the PTZ camera will be triggered to adjust the focal length, restore the PTZ camera to a default position, stop adjusting the focal length, and repeat the panoramic detection of the wide-angle lens.

[0143] In this embodiment, by introducing the unlocking gesture, the speaker can restore the default position of the PTZ camera at any time, thereby providing a more flexible operation mode for the PTZ camera.

[0144] Based on the same concept, in one embodiment, Fig. 9 As shown, the present invention also provides a PTZ camera, which is applied to the static gesture interaction method described in any of the above embodiments. The PTZ camera 900 includes: a wide-angle lens 911, a human body detection module 912, a speaker tracking module 913, a human body key point detection module 914, a hand extension module 915, a gesture key point detection module 916, a gesture distinction module 917, a human body position calculation module 918, a focus tracking calculation module 919 and a focus tracking lens 920. Among them:

[0145] The wide-angle lens 911 is arranged at the bottom of the PTZ camera, and is used to perform real-time panoramic detection of the conference venue, obtain the video stream of the conference venue, and detect the participants in the conference venue in real time.

[0146] The human body detection module 912 is used to perform human body detection in the video stream and obtain human body range frames of several participants. Specifically, human body detection is performed in the video stream through a computer vision algorithm, and N human body range frames are generated for the detected participants in the form of pixel coordinates under the photo (there are N participants in the video stream), and human body range frames of several participants are obtained, and each participant has a specified human body range frame (x, y, h, w), where x is the horizontal coordinate, y is the vertical coordinate, h is the height, and w is the width.

[0147] More specifically, human body detection is performed on the current frame of the video stream through a computer vision algorithm. After the human body is detected, the current frame of the video stream is cropped to obtain a human body image, and a human body range frame (x, y, h, w) is generated for the human body image in the form of pixel coordinates under the photo. If N participants are detected in the video stream, N human body range frames (x, y, h, w) are generated, thereby obtaining the human body range frames of several participants, wherein x is the horizontal coordinate, y is the vertical coordinate, h is the height, and w is the width.

[0148] The speaker tracking module 913 is used to track the speaker across devices or across time after identifying the speaker among a number of conference participants.

[0149] The speaker tracking module 913 uses visual algorithms to match images or videos of speakers across devices or time. That is, given a query speaker image, the same speaker can be retrieved from the image library under the wide-angle lens and the tracking lens, or the same speaker can be tracked when the speaker walks out of the wide-angle lens and reappears.

[0150] Specifically, it is determined whether the gesture categories of several participants are subsets of the preset trigger static gesture library. If it is determined that the gesture category of one of the participants is a subset of the preset trigger static gesture library, the participant is identified as a speaker, the human body range frame corresponding to the participant is found, the mark is set as the speaker's human body range frame, and tracking zoom is triggered to track the speaker across devices or across time; otherwise, tracking zoom is not triggered, and the panoramic detection of the wide-angle lens is repeated. In this way, accurate speaker identification and tracking are achieved.

[0151] When tracking zoom is triggered, the wide-angle lens automatic tracking function of the PTZ camera and the automatic zoom function of the focus lens will be triggered. The wide-angle lens of the PTZ camera will automatically track the participants who have triggered the zoom gesture or have the focus locked, and track the speaker when the speaker reappears after walking out of the wide-angle lens. If the speaker does not walk out of the wide-angle lens in a certain frame sequence, the human body range frame of the previous frame is expanded by the first external expansion ratio as the output result of the human body range frame of the current frame to reduce the computing power occupied by the algorithm.

[0152] The human body key point detection module 914 is used to identify the human body key point information of several conference participants according to the human body range frames of several conference participants through a preset algorithm.

[0153] Specifically, the human key point detection module 914 is a pre-task for human motion recognition, behavior analysis, human-computer interaction, etc. In general, human key point detection can be subdivided into single / multi-person key point detection and 2D / 3D key point detection. After completing the key point detection, key point tracking will also be performed, which is also called human posture tracking.

[0154] The preset algorithm includes a deep learning algorithm, which is used to identify the human body range frames of several participants, obtain the human body key point information of several participants on the image plane of the current frame of the video stream corresponding to N participants (there are N participants in the video stream, so there are N human body key point information), and realize accurate identification of the human body key point information of the participants.

[0155] like Figure 3 As shown in FIG. 1 , the key points of the human body include 17 joints, namely nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles. The key point information of the human body includes the above 17 joint information.

[0156] The speaker tracking module 913 outputs the human body range frame of the speaker in the previous frame and the image obtained by cropping the current frame of the first video stream of the wide-angle lens, which are input into the human body key point detection module 914. The human body key point detection module 914 recognizes the human body key point information of several participants based on the human body range frames of several participants through a preset algorithm.

[0157] The hand expansion module 915 is used to predict the hand positions of several participants according to the human body key point information or historical frame hand range frame information of several participants, and output the hand range frames of several participants.

[0158] Specifically, for each participant's body key point information, the hand range box (x_1, y_1, h_1, w_1) of each participant is determined (calculated) according to the preset hand position prediction logic, where x_1 is the horizontal coordinate, y_1 is the vertical coordinate, h_1 is the height, and w_1 is the width.

[0159] Among them, the preset hand position prediction logic includes:

[0160] A. Through the hand range frame of the historical frame, extract the movement speed, movement trajectory and other information of the hand in the historical frame, predict the current frame position through the hand range frame of the previous 5 frames, and obtain the first predicted hand range frame.

[0161] B. The human body key point information of the participants is obtained through the human body key point detection module 914, including the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, left and right ankles and other human body key points. The hand center distance is taken on the extension line of the "left elbow-left arm" / "right elbow-right arm" key point, and the circle of the hand diameter is taken, and its circumscribed rectangle is used as the second predicted hand range frame. The first predicted hand range frame and the second predicted hand range frame are positionally fused. When the IOU (Intersection over Union) value of the first predicted hand range frame and the second predicted hand range frame is large, the overlapping area of ​​the two is taken for the second outward expansion ratio expansion and then output as the result of the hand range frame of the current frame. Among them, IOU (Intersection over Union) is used to describe the degree of overlap of two frames. The larger the overlapping area, the larger the IOU value.

[0162] C. When the IOU value between the first predicted hand range box and the second predicted hand range box is less than the threshold, if the credibility of the human key point information is lower than the human credibility threshold, the first predicted hand range box is used to perform a second outward expansion ratio expansion and is output as the result of the hand range box of the current frame; otherwise, the second predicted hand range box is used to perform a second outward expansion ratio expansion and is output as the result of the hand range box of the current frame.

[0163] The gesture key point detection module 916 is used to obtain gesture key point information of several conference participants according to the hand range frames of several conference participants.

[0164] Specifically, Figure 5 As shown in the figure, 21 key points of hand gesture are defined, which describe the wrist, finger end joints, finger tip joints, finger proximal joints, and finger distal joints respectively.

[0165] By detecting the above-mentioned gesture key points at specific positions of the palm and fingers, the position and direction information of the palm and fingers can be obtained, and the positions of the palm and fingers can be located.

[0166] In the gesture key point detection module 916, by detecting gesture key points at specific positions of the hand and fingers, the position and direction information of the palm and fingers can be obtained, and the position of the palm and fingers can be located.

[0167] The gesture distinguishing module 917 is used to obtain gesture categories corresponding to the participants according to the gesture key point information of the participants.

[0168] Specifically, according to the gesture key point information of several participants, the state of the finger joints is calculated (for example: straight, curled, relaxed), the palm direction is calculated, and the current gesture is determined by combining multiple finger states to determine whether it is included in the preset trigger static gesture library, and the gesture classification and palm direction results are returned, so as to obtain the gesture categories corresponding to several participants by identifying the gesture key point information. Among them, the preset trigger static gesture library includes at least the following static gestures: five fingers open, thumbs up, ok, scissors hand, fist.

[0169] The human body position calculation module 918 is used to perform geometric calculations based on the speaker's human body key point information and the parameters of the wide-angle lens, and obtain the speaker's posture information relative to the wide-angle lens of the PTZ camera by substituting the standard human body parameter model, thereby determining the speaker's human body position information.

[0170] The focus tracking calculation module 919 is used to determine the intrinsic parameters and extrinsic parameters of the virtual camera according to the body position information of the speaker using the camera model, and convert the intrinsic and extrinsic parameters of the virtual camera into focus tracking lens parameters of the PTZ camera.

[0171] The tracking lens 920 is arranged on the top of the PTZ camera, and is used to automatically adjust the focal length and the pan / tilt rotation angle according to the tracking lens parameters of the PTZ camera, automatically realize the zoom operation of the PTZ camera, and accurately focus on the focal length required by the speaker.

[0172] In this embodiment, a PTZ camera is provided to collect the video stream of the conference site in real time, identify the key point information of the human body of several participants from the video stream; determine the gesture categories of several participants based on the key point information of the human body of several participants; identify the speaker based on the gesture categories of several participants, and track the speaker across devices or across time; determine the body position information of the speaker based on the identified speaker; automatically adjust the focus lens parameters of the focus lens of the PTZ camera based on the body position information of the speaker, and automatically realize the zoom operation of the PTZ camera. Thus, the key point information and gesture categories of several participants are identified from the video stream through the key point recognition technology of the human body; the speaker is identified from the gesture categories of several participants through the gesture recognition technology, and accurate speaker identification and tracking are realized; and the gesture recognition technology recognizes the speaker's gesture, automatically triggers the zoom function of the PTZ camera, and automatically realizes the adjustment of the focal length. Thus, the PTZ camera combines the human key point recognition and gesture recognition technology, and enables the PTZ camera to have gesture interaction function and automatic focus tracking capability, realizes static gesture interaction based on the PTZ camera, and automatically realizes the zoom operation of the PTZ camera, without the need for special personnel to manually operate the focus tracking lens zoom. The operation method is natural, intuitive, convenient, and responsive, which improves the interaction efficiency and user experience. This can solve the problem that the existing PTZ camera adopts cumbersome manual operation, is not intuitive and convenient, and has slow response, which affects the interaction efficiency.

[0173] It should be noted that the above-mentioned PTZ camera embodiment and method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the PTZ camera embodiment, which will not be repeated here.

[0174] Based on the same concept, in one embodiment, the present invention also provides a PTZ camera, such as Fig.10 As shown, the PTZ camera 900 includes: a memory 902, a processor 901, and one or more computer programs stored in the memory 902 and executable on the processor 901. The memory 902 and the processor 901 are coupled together via a bus system 903. When the one or more computer programs are executed by the processor 901, the following steps of a static gesture interaction method provided in an embodiment of the present invention are implemented:

[0175] S1, collect the video stream of the conference site in real time, and identify the key points of the human body of several participants from the video stream;

[0176] S2. determining the gesture categories of the participants according to the key point information of the participants;

[0177] S3, identifying the speaker based on the gesture categories of several participants, and tracking the speaker across devices or across time;

[0178] S4, determining the body position information of the speaker according to the identified speaker;

[0179] S5. Automatically adjust the focus lens parameters of the focus lens of the PTZ camera according to the body position information of the speaker, and automatically implement the zoom operation of the PTZ camera.

[0180] The method disclosed in the above embodiment of the present invention can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the processor 901. The processor 901 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 901 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 902. The processor 901 reads the information in the memory 902 and completes the steps of the above method in combination with its hardware.

[0181] It can be understood that the memory 902 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory or other memory technology, a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD) or other optical disk storage, a magnetic cassette, a magnetic tape, a magnetic disk storage or other magnetic storage device; the volatile memory can be a random access memory (RAM), and by way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM Memory), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0182] It should be noted that the above-mentioned PTZ camera embodiment and method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the PTZ camera embodiment, which will not be repeated here.

[0183] In addition, in an exemplary embodiment, an embodiment of the present invention further provides a computer storage medium, specifically a computer-readable storage medium, for example, including a memory 902 storing a computer program, wherein one or more programs of a static gesture interaction method are stored on the computer storage medium, and when the one or more programs of the static gesture interaction method are executed by the processor 901, the following steps of a static gesture interaction method provided in an embodiment of the present invention are implemented:

[0184] S1, collect the video stream of the conference site in real time, and identify the key points of the human body of several participants from the video stream;

[0185] S2. determining the gesture categories of the participants according to the key point information of the participants;

[0186] S3, identifying the speaker based on the gesture categories of several participants, and tracking the speaker across devices or across time;

[0187] S4, determining the body position information of the speaker according to the identified speaker;

[0188] S5. Automatically adjust the focus lens parameters of the focus lens of the PTZ camera according to the body position information of the speaker, and automatically implement the zoom operation of the PTZ camera.

[0189] It should be noted that the static gesture interaction method program embodiment on the above-mentioned computer-readable storage medium and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the embodiment of the above-mentioned computer-readable storage medium, which will not be repeated here.

[0190] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0191] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the above embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A static gesture interaction method, characterized in that: Applied to a PTZ camera, the static gesture interaction method includes: Collect the video stream of the conference site in real time, and identify the key points of the human bodies of several participants from the video stream; Determine the gesture categories of the participants according to the key point information of the participants; Identify the speaker based on the gesture categories of several participants, and track the speaker across devices or across time; Determining body position information of the speaker according to the identified speaker; According to the body position information of the speaker, the focus lens parameters of the focus lens of the PTZ camera are automatically adjusted to automatically implement the zoom operation of the PTZ camera.

2. The static gesture interaction method according to claim 1, characterized in that: The real-time acquisition of the video stream of the conference site and identification of key body point information of several participants from the video stream include: Get the video stream of the venue from the wide-angle lens of the PTZ camera; Performing human body detection on the video stream to obtain human body range frames of several participants; According to the human body range frames of several participants, the human body key point information of several participants is identified through a preset algorithm.

3. The static gesture interaction method according to claim 1, characterized in that: Determining the gesture categories of the plurality of participants according to the body key point information of the plurality of participants includes: According to the human body key point information of several participants, the hand positions of several participants are predicted, and the hand range frames of several participants are output; According to the hand range frames of the participants, the key point information of the gestures of the participants is obtained; According to the gesture key point information of several participants, gesture categories corresponding to the several participants are obtained.

4. The static gesture interaction method according to claim 2, characterized in that: The method of identifying a speaker according to the gesture categories of the plurality of participants and tracking the speaker across devices or across time includes: It is determined whether the gesture categories of several participants are subsets of a preset trigger static gesture library. If it is determined that the gesture category of one of the participants is a subset of the preset trigger static gesture library, the participant is identified as a speaker, a human body range frame corresponding to the participant is found, the mark is set as the speaker's human body range frame, and tracking zoom is triggered to track the speaker across devices and / or across time.

5. The static gesture interaction method according to claim 4, characterized in that: The step of determining the body position information of the speaker according to the identified speaker includes: After triggering the tracking of the speaker, determining whether the speaker's gesture is a triggering gesture through gesture recognition based on the key point information of the speaker's body; If it is determined that the speaker's gesture is a trigger gesture, further determining whether the trigger gesture is a focus locking gesture; If it is determined that the trigger gesture is a focus locking gesture, the body position information of the speaker is determined according to the body key point information of the speaker.

6. The static gesture interaction method according to claim 1, characterized in that: The method of automatically adjusting the focus lens parameters of the focus lens of the PTZ camera according to the body position information of the speaker to automatically implement the zoom operation of the PTZ camera includes: Determining intrinsic parameters and extrinsic parameters of a virtual camera using a camera model according to the body position information of the speaker; Converting the internal and external parameters of the virtual camera into focus lens parameters of the PTZ camera; According to the focus tracking lens parameters of the PTZ camera, the focal length of the focus tracking lens and the pan / tilt rotation angle of the PTZ camera are automatically adjusted to automatically implement the zoom operation of the PTZ camera.

7. The static gesture interaction method according to claim 1, characterized in that: The static gesture interaction method further includes: restoring the PTZ camera to a default position and stopping the adjustment of the focal length according to the unlocking gesture of the speaker.

8. A PTZ camera, characterized in that: The static gesture interaction method applied to any one of claims 1 to 7, wherein the PTZ camera comprises a wide-angle lens, a human body detection module, a speaker tracking module, a human body key point detection module, a hand extension module, a gesture key point detection module, a gesture differentiation module, a human body position calculation module, a focus tracking calculation module and a focus tracking lens; wherein: The wide-angle lens is arranged at the bottom of the PTZ camera and is used to perform real-time panoramic detection of the venue, obtain the video stream of the venue, and detect the participants in the venue in real time; The human body detection module is used to perform human body detection on the video stream to obtain human body range frames of several participants; The speaker tracking module is used to track the speaker across devices or across time after identifying the speaker among a number of conference participants; The human body key point detection module is used to identify the human body key point information of several participants according to the human body range frames of several participants through a preset algorithm; The hand expansion module is used to predict the hand positions of several participants according to the human body key point information or the hand range frame information of the historical frames, and output the hand range frames of several participants; The gesture key point detection module is used to obtain the gesture key point information of several participants according to the hand range frames of several participants; The gesture distinguishing module is used to obtain gesture categories corresponding to a number of participants according to the gesture key point information of a number of participants; The human body position calculation module is used to perform geometric calculations based on the human body key point information of the speaker and the parameters of the wide-angle lens, obtain the posture information of the speaker relative to the wide-angle lens of the PTZ camera by substituting into a standard human body parameter model, and determine the human body position information of the speaker; The focus tracking calculation module is used to determine the intrinsic parameters and extrinsic parameters of the virtual camera using the camera model according to the body position information of the speaker, and convert the intrinsic and extrinsic parameters of the virtual camera into focus tracking lens parameters of the PTZ camera; The tracking lens is arranged on the top of the PTZ camera, and is used to automatically adjust the focal length and the pan / tilt rotation angle according to the tracking lens parameters of the PTZ camera, so as to automatically realize the zoom operation of the PTZ camera.

9. A PTZ camera, characterized in that: include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the static gesture interaction method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium stores a program of the static gesture interaction method, and when the program of the static gesture interaction method is executed by the processor, the static gesture interaction method according to any one of claims 1 to 8 is implemented.

Images