Camera self-adaptive adjustment method based on multi-target detection

Abstract

The application discloses a camera self-adaptive adjustment method based on multi-target detection. The method detects human shape, face, palm gesture and OK gesture in the input video frame by using a lightweight multi-target detection model, and carries out self-adaptive discrimination according to the detection result, and continues to adjust the camera according to the discrimination result. First, a lightweight multi-target neural network detection model is constructed, the video frame is input into the multi-target detection network model after normalization, the model prediction information is divided into a set, the detection result output by the multi-target detection network model is obtained, the self-adaptive discrimination module outputs a discrimination instruction, and the camera is adjusted so that the target is in the middle position of the camera picture. The method of the application does not need to retain the information of the previous frame, and can adjust the camera only by relying on the detection result of the current frame. The application has the advantages of high detection sensitivity and convenient use, and can be widely applied to live broadcast, outdoor photography, monitoring and other scenes.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a camera adaptive adjustment method based on multi-target detection. Background Technology

[0002] In recent years, the live video streaming industry has been developing rapidly, with live streaming services continuously spreading to various sectors. Typically, before a live stream, personnel will fix and adjust the camera based on the streamer's position. However, during the live stream, if the subject moves, the equipment needs to be manually readjusted to position the subject in the optimal position for high-quality video. Manual adjustments either require additional personnel or the streamer to make the adjustments themselves, leading to increased costs, inconvenience, and inconsistent image quality, resulting in a poor live stream experience. Similar issues exist in scenarios such as outdoor selfies.

[0003] Existing camera adaptive adjustment methods mainly fall into two categories. One category is non-neural network-based detection and tracking methods, which determine the target's movement direction and position by filtering and extracting feature information from previous and subsequent frames. While these methods have low computational requirements, they suffer from insensitive adjustments and errors, especially in complex environments where performance degradation is more pronounced. The other category is neural network-based detection and tracking methods. These methods leverage the powerful feature extraction capabilities of neural networks to accurately detect the target's position, but they require high computational power. Furthermore, current detection methods mostly detect single objects, failing to fully utilize the advantages of neural networks. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a camera adaptive adjustment method based on multi-target detection. This method employs a neural network model to simultaneously detect multiple targets, including human figures, faces, "OK" gestures, and "palm" gestures. It then assigns face and gesture targets based on the human figure target, ensuring accurate gesture control in multi-person scenarios. Furthermore, the model utilizes a lightweight structure, enabling real-time detection and meeting application requirements.

[0005] The method of this invention is as follows:

[0006] Step (1) Construct a lightweight multi-target neural network detection model, perform deep learning training on four types of targets, generate a trained multi-target detection network model, and solidify the multi-target detection network model; the four types of targets are human figures, human faces, OK gestures and Palm gestures, OK gestures indicate adjusting the camera, and Palm gestures indicate stopping adjusting the camera.

[0007] The lightweight multi-target neural network detection model includes a feature extraction module and a target prediction module.

[0008] The feature extraction module includes: a first convolutional layer Pcbr1 and N lightweight residual stacks LB. The N lightweight residual stacks LB are concatenated and then concatenated with the first convolutional layer Pcbr1. The outputs of the last three lightweight residual stacks LB serve as the third-level output of the feature extraction module, where N∈[5,6,7,8,9,10].

[0009] The first convolutional layer Pcbr1 consists of a cascade of 1x1 convolutional layers, batch normalization (BN), and a nonlinear mapping layer ReLU.

[0010] The lightweight residual stack (LB) consists of seven convolutional layers, of which:

[0011] The second convolutional layer Dcbr2 consists of a cascaded 3x3 separable convolution with a stride of 2, batch normalization (BN), and a non-linear mapping layer ReLU.

[0012] The third convolutional layer Pcbr3, the fourth convolutional layer Pcbr4, the fifth convolutional layer Pcbr5, the seventh convolutional layer Pcbr7, and the eighth convolutional layer Pcbr8 are the same as the first convolutional layer Pcbr1 except for the input and output channels.

[0013] The sixth convolutional layer, Dcbr6, consists of a cascade of a 3x3 separable convolution with a stride of 1, a batch normalized (BN) layer, and a nonlinear mapping layer, ReLU.

[0014] The second convolutional layer Dcbr2 is cascaded with the third convolutional layer Pcbr3, and then connected to the fourth convolutional layer Pcbr4 and the fifth convolutional layer Pcbr5 respectively. The fifth convolutional layer Pcbr5, the sixth convolutional layer Dcbr6, and the seventh convolutional layer Pcbr7 are cascaded step by step. The seventh convolutional layer Pcbr7 and the fourth convolutional layer Pcbr4 are concatenated and then cascaded with the eighth convolutional layer Pcbr8. Finally, the output of the eighth convolutional layer Pcbr8 of the three lightweight residual stacks LB is used as one output.

[0015] The target prediction module includes three convolutional layers: the ninth convolutional layer Pcbs9; the tenth and eleventh convolutional layers Pcbs10 and Pcbs11 are both composed of 1x1 convolutions, batch normalization (BN), and sigmoid; the three-level features extracted by the feature extraction module are respectively input into the three convolutional layers of the target prediction module, and the model prediction information is obtained by concatenation and combination.

[0016] Four types of target data are input into a lightweight multi-target neural network detection model and trained using deep learning to obtain a converged multi-target detection network model. Then, batch normalization (BN) in the model is merged into the corresponding convolutional convolution, and the floating-point parameters of the model are quantized into 8-bit format, thus completing the solidification of the multi-target detection network model.

[0017] Step (2) Acquire video frames, normalize the video frame data, and input it into the fixed multi-object detection network model to obtain model inference information. The model inference information includes multiple predicted target boxes, and the i-th predicted target box is in the format [class i ,x i 1,y i 1,x i 2,y i 2,conf i ], where class i Indicates the category of the target, conf i Indicates the confidence level of the target, [x i 1,y i 1,x i 2,y i 2] represents the coordinate information of the target; data normalization is to normalize the video frame data to [0,1]. After the input frame data img_in is normalized, the output data img_out is generated, then img_out = img_in / 255.0.

[0018] Step (3) Sort the predicted target boxes of the model prediction information from high to low according to the confidence level (conf), and then divide them into 4 sets according to the class: human shape set H, human face set F, OK gesture set O and palm gesture set P.

[0019] Delete the predicted bounding boxes of human figures in the human figure set H with a confidence level less than the set threshold th_h, delete the predicted bounding boxes of faces in the face set F with a confidence level less than the set threshold th_f, delete the predicted bounding boxes of OK gestures in the OK gesture set O with a confidence level less than the set threshold th_o, and delete the predicted bounding boxes of Palm gestures in the Palm gesture set P with a confidence level less than the set threshold th_p.

[0020] Let the targets belonging to the same person from the human figure set H, the face set F, the OK gesture set O, and the Palm gesture set P be the target set R for that person; the target set R for the j-th person is... j =[h j ,f j ,o j / p j ], h j The humanoid target of the j-th person in the humanoid set H, f j The face target of the j-th person in the face set F, o j Let p be the target of the OK gesture for the j-th person in the OK gesture set O. j Let P be the palm gesture target of the j-th person in the palm gesture set, and / denotes "or"; the target set R allows face targets or gesture targets to be empty; the set sequence [R1,...,R] is...j …,R J This refers to the detection result output by the multi-target detection network model.

[0021] Step (4) Input the detection results output by the multi-target detection network model into the adaptive discrimination module; the adaptive discrimination module outputs a discrimination instruction. If the instruction is to adjust, then execute step (5). If the instruction is not to adjust, then return to step (2) and execute the next frame.

[0022] Step (5) Adjust the camera so that the target is in the center of the camera frame, as follows:

[0023] Given the face coordinates [x1, y1, x2, y2] corresponding to the input adaptive adjustment set, divide the input video frame into M uniform stripes in the horizontal dimension. The range corresponding to the center position is then... W represents the width of the corresponding video frame data. Indicates rounding down;

[0024] The coordinates of the horizontal center of the face in the image That is, at the position of the m-th slat out of M slats. Rotate the camera and pan the image to center the face in the frame; adjust the number of bars. If x_mov > 0, the camera rotates to the left; if x_mov < 0, the camera rotates to the right; if x_mov = 0, the camera does not rotate.

[0025] Step (6) returns to step (2) and executes the next frame.

[0026] Furthermore, in step (4), the adaptive discrimination module outputs different discrimination instructions according to the four scenarios:

[0027] Scenario 1: If the set sequence [R1,…,R] j …,R J If there is a target set R in the set where neither face target nor gesture target is empty, then select the target set R with the highest confidence of the corresponding humanoid target from the target set R where neither face target nor gesture target is empty, and use it as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the OK gesture, then set the adaptive adjustment flag ada_flag to True and save it, issue an adjustment command, output the face coordinates [x1,y1,x2,y2] corresponding to the adaptive adjustment set, and execute step (5); if the gesture target in the adaptive adjustment set is the Palm gesture, then set the adaptive adjustment flag ada_flag to False and save it, issue a return command, return to step (2), and execute the next frame;

[0028] Scenario 2, if the set sequence [R1,…,R]j …,R J There is no target set R in which both face targets and gesture targets are not empty, but there is a target set R in which face targets are empty and gesture targets are not empty. Take the target set R with the highest confidence of the corresponding human-shaped target from the target set R in which face targets are empty and gesture targets are not empty, and use it as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the OK gesture, set the adaptive adjustment flag ada_flag to True and save it, and issue a return command; if the gesture target in the adaptive adjustment set is the Palm gesture, set the adaptive adjustment flag ada_flag to False and save it, and issue a return command; return to step (2) and execute the next frame;

[0029] Scenario 3, if the set sequence [R1,…,R] j …,R J There is no target set R in which both face and gesture targets are not empty, but there is a target set R where gesture targets are empty and face targets are not empty. If the current adaptive adjustment flag ada_flag is False, issue a return instruction, return to step (2), and execute the next frame; if the current adaptive adjustment flag ada_flag is True, output the face coordinates [x1, y1, x2, y2] corresponding to the adaptive adjustment set, and execute step (5);

[0030] Scenario 4, if the set sequence [R1,…,R] j …,R J If the face target and gesture target in the target set R are both empty, a return command is issued, returning to step (2) and executing the next frame.

[0031] This invention uses gestures to control camera adjustments. The "OK" gesture indicates camera adjustment, while the "palm" gesture indicates stopping adjustment. Gesture detection and recognition fully free up hands, enabling the machine to respond to human commands and move accordingly. This method not only features a lightweight model, making it easy to deploy on the device, but also boasts high detection accuracy, sensitive adjustment, and real-time capability. It can automatically track targets in scenarios such as chat, live streaming, monitoring, and selfies, ensuring the broadcaster remains centered on the screen for a better viewing experience. Attached Figure Description

[0032] Figure 1 This is a simplified flowchart of the present invention;

[0033] Figure 2 A schematic diagram of a lightweight multi-target neural network detection model;

[0034] Figure 3 This is a schematic diagram of the lightweight residual stack (LB) in the feature extraction module. Detailed Implementation

[0035] like Figure 1 As shown, the adaptive camera adjustment method based on multi-target detection has the following steps:

[0036] Step (1) Construct a lightweight multi-target neural network detection model, perform deep learning training on four types of targets, generate a trained multi-target detection network model, and then solidify the multi-target detection network model. The four targets are human figures, faces, OK gestures, and Palm gestures. The OK gesture indicates adjusting the camera, and the Palm gesture indicates stopping the camera adjustment.

[0037] The lightweight multi-object neural network detection model includes a feature extraction module and an object prediction module, such as... Figure 2 As shown, the two dashed boxes represent two modules respectively.

[0038] The feature extraction module includes: a first convolutional layer Pcbr1 and N lightweight residual stacks LB, where N∈[5,6,7,8,9,10], and in this embodiment N=7; the N lightweight residual stacks LB are concatenated and then concatenated with the first convolutional layer Pcbr1, and the outputs of the last three lightweight residual stacks LB serve as the third-level output of the feature extraction module;

[0039] The first convolutional layer Pcbr1 consists of a cascade of 1x1 pointwise convolutions, batch normalization (BN), and a nonlinear mapping layer ReLU.

[0040] like Figure 3 As shown, the lightweight residual stack LB consists of seven convolutional layers, wherein:

[0041] The second convolutional layer, Dcbr2, consists of a cascade of a 3x3 separable convolution (Depthwise conv) with a stride of 2, batch normalization (BN), and a nonlinear mapping layer, ReLU.

[0042] The third convolutional layer Pcbr3, the fourth convolutional layer Pcbr4, the fifth convolutional layer Pcbr5, the seventh convolutional layer Pcbr7, and the eighth convolutional layer Pcbr8 are the same as the first convolutional layer Pcbr1 except for the input and output channels.

[0043] The sixth convolutional layer, Dcbr6, consists of a cascade of a 3x3 separable convolution (Depthwise conv) with a stride of 1, batch normalization (BN), and a nonlinear mapping layer, ReLU.

[0044] The second convolutional layer Dcbr2 is cascaded with the third convolutional layer Pcbr3, then connected to the fourth convolutional layer Pcbr4 and the fifth convolutional layer Pcbr5. The fifth convolutional layer Pcbr5, the sixth convolutional layer Dcbr6, and the seventh convolutional layer Pcbr7 are cascaded sequentially. The seventh convolutional layer Pcbr7 and the fourth convolutional layer Pcbr4 are then concatenated with the eighth convolutional layer Pcbr8. Finally, the outputs of the eighth convolutional layer Pcbr8 of the three lightweight residual stacks LB are each used as one output.

[0045] The target prediction module includes three convolutional layers: the ninth convolutional layer Pcbs9; the tenth and eleventh convolutional layers Pcbs10 and Pcbs11 are both composed of 1x1 pointwise convolutions, batch normalization (BN), and sigmoid; the three-level features extracted by the feature extraction module are input into the three convolutional layers of the target prediction module, and the model prediction information is obtained by concatenation and combination.

[0046] Four types of target data are input into a lightweight multi-target neural network detection model for deep learning training, resulting in a converged multi-target detection network model. Then, batch normalization (BN) in the model is merged into the corresponding convolutional (conv) layers. Finally, the model's floating-point parameters are quantized into 8-bit format, thus completing the solidification of the multi-target detection network model.

[0047] Step (2) Acquire video frames, normalize the video frame data, and input it into the fixed multi-object detection network model to obtain model inference information. The model inference information includes multiple predicted target boxes, and the i-th predicted target box is in the format [class i ,x i 1,y i 1,x i 2,y i 2,conf i ], where class i Indicates the category of the target, conf i Indicates the confidence level of the target, [x i 1,y i 1,x i 2,y i 2] represents the coordinate information of the target. Data normalization normalizes the video frame data to [0,1]. After normalization, the input frame data img_in is output as img_out, then img_out = img_in / 255.0.

[0048] Step (3) Sort the predicted target boxes of the model prediction information from high to low confidence level (conf), and then divide them into 4 sets according to the class (class): human figure set H, human face set F, OK gesture set O, and palm gesture set P.

[0049] The predicted bounding boxes of human figures in the human figure set H with a confidence level less than the set threshold th_h are deleted. The predicted bounding boxes of faces in the face set F with a confidence level less than the set threshold th_f are deleted. The predicted bounding boxes of OK gestures in the OK gesture set O with a confidence level less than the set threshold th_o are deleted. The predicted bounding boxes of Palm gestures in the Palm gesture set P with a confidence level less than the set threshold th_p are deleted. In this embodiment, the set thresholds th_h = 0.55, th_f = 0.50, th_o = 0.45, and th_p = 0.45.

[0050] The targets belonging to the same person from the human figure set H, the face set F, the OK gesture set O, and the palm gesture set P are defined as the target set R for this person, as follows:

[0051] S301, For h in H:

[0052] S302. Add h to R;

[0053] S303, for f in F:

[0054] S304, if the coordinates in f are located in h:

[0055] S305. Add f to R and remove f from F;

[0056] S306、 for o in O:

[0057] S307, if the coordinates in o are located in h:

[0058] S308, if p exists in R:

[0059] S309. If the confidence level corresponding to p is less than the confidence level corresponding to o, then remove p from R, add o to R, and remove o from O; otherwise, remove o from O.

[0060] S310, else:

[0061] S311. Add o to R and remove o from O;

[0062] S312, for p in P:

[0063] S313, if the coordinates in p are located in h:

[0064] S314, if o exists in R:

[0065] S315. If the confidence level corresponding to o is less than the confidence level corresponding to p, then o is removed from R, p is added to R, and p is removed from P; if the confidence level corresponding to o is greater than p, then p is removed from R; otherwise, p is removed from P.

[0066] S316, else:

[0067] S317. Add p to R and remove p from P;

[0068] Finally, the target set R is obtained.

[0069] The goal set of the j-th person is R. j =[h j ,f j ,o j / p j ], h j The humanoid target of the j-th person in the humanoid set H, f j The face target of the j-th person in the face set F, o j Let p be the target of the OK gesture for the j-th person in the OK gesture set O. j Let R be the palm gesture target of the j-th person in the palm gesture set P, where / denotes "or"; the target set R allows either face targets or gesture targets to be empty. The set sequence [R1, ..., R] j …,R J This refers to the detection result output by the multi-target detection network model.

[0070] Step (4) Input the detection results output by the multi-target detection network model into the adaptive discrimination module; The adaptive discrimination module outputs a discrimination instruction. If the instruction is to adjust, then execute step (5). If the instruction is not to adjust, then return to step (2) and execute the next frame.

[0071] The adaptive discrimination module outputs different discrimination instructions based on the four scenarios:

[0072] Scenario 1: If the set sequence [R1,…,R] j …,R JIf there is a target set R in the set where neither face target nor gesture target is empty, then select the target set R with the highest confidence of the corresponding humanoid target from the target set R where neither face target nor gesture target is empty, and use it as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the OK gesture, then set the adaptive adjustment flag ada_flag to True and save it, issue an adjustment command, output the face coordinates [x1,y1,x2,y2] corresponding to the adaptive adjustment set, and execute step (5); if the gesture target in the adaptive adjustment set is the Palm gesture, then set the adaptive adjustment flag ada_flag to False and save it, issue a return command, return to step (2), and execute the next frame;

[0073] Scenario 2, if the set sequence [R1,…,R] j …,R J There is no target set R in which both face targets and gesture targets are not empty, but there is a target set R in which face targets are empty and gesture targets are not empty. Take the target set R with the highest confidence of the corresponding human-shaped target from the target set R in which face targets are empty and gesture targets are not empty, and use it as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the OK gesture, set the adaptive adjustment flag ada_flag to True and save it, and issue a return command; if the gesture target in the adaptive adjustment set is the Palm gesture, set the adaptive adjustment flag ada_flag to False and save it, and issue a return command; return to step (2) and execute the next frame;

[0074] Scenario 3, if the set sequence [R1,…,R] j …,R J There is no target set R in which both face and gesture targets are not empty, but there is a target set R where gesture targets are empty and face targets are not empty. If the current adaptive adjustment flag ada_flag is False, issue a return instruction, return to step (2), and execute the next frame; if the current adaptive adjustment flag ada_flag is True, output the face coordinates [x1, y1, x2, y2] corresponding to the adaptive adjustment set, and execute step (5);

[0075] Scenario 4, if the set sequence [R1,…,R] j …,R J If the face target and gesture target in the target set R are both empty, a return command is issued, returning to step (2) and executing the next frame.

[0076] Step (5) Adjust the camera so that the target is in the center of the camera frame, as follows:

[0077] Given the face coordinates [x1, y1, x2, y2] corresponding to the input adaptive adjustment set, divide the input video frame into M uniform stripes in the horizontal dimension. The range corresponding to the center position is then... W represents the width of the corresponding video frame data. Indicates rounding down;

[0078] The coordinates of the horizontal center of the face in the image That is, at the position of the m-th slat out of M slats. Rotate the camera and pan the image to center the face in the frame; adjust the number of bars. If x_mov > 0, the camera rotates to the left; if x_mov < 0, the camera rotates to the right; if x_mov = 0, the camera does not rotate.

[0079] Step (6) returns to step (2) and executes the next frame.

Claims

1. A camera adaptive adjustment method based on multi-target detection, characterized in that: Step (1) Construct a lightweight multi-target neural network detection model, perform deep learning training on four types of targets, generate a trained multi-target detection network model, and solidify the multi-target detection network model; the four types of targets are human figures, faces, OK gestures and Palm gestures, OK gestures indicate adjusting the camera, and Palm gestures indicate stopping adjusting the camera; The lightweight multi-target neural network detection model includes a feature extraction module and a target prediction module; The feature extraction module includes: a first convolutional layer Pcbr1 and... A lightweight residual stack LB, The lightweight residual stacks (LB) are cascaded together and then cascaded together with the first convolutional layer (Pcbr1). The outputs of the last three lightweight residual stacks (LB) serve as the third-level outputs of the feature extraction module. The first convolutional layer Pcbr1 consists of a cascade of 1x1 convolutional layers, batch normalization (BN), and a nonlinear mapping layer ReLU. The lightweight residual stack (LB) consists of seven convolutional layers, of which: The second convolutional layer Dcbr2 consists of a cascaded 3x3 separable convolution with a stride of 2, batch normalization (BN), and a non-linear mapping layer ReLU. The third convolutional layer Pcbr3, the fourth convolutional layer Pcbr4, the fifth convolutional layer Pcbr5, the seventh convolutional layer Pcbr7, and the eighth convolutional layer Pcbr8 are the same as the first convolutional layer Pcbr1 except for the input and output channels. The sixth convolutional layer, Dcbr6, consists of a cascade of a 3x3 separable convolution with a stride of 1, a batch normalized (BN) layer, and a nonlinear mapping layer, ReLU. The second convolutional layer Dcbr2 is cascaded with the third convolutional layer Pcbr3, and then connected to the fourth convolutional layer Pcbr4 and the fifth convolutional layer Pcbr5 respectively. The fifth convolutional layer Pcbr5, the sixth convolutional layer Dcbr6, and the seventh convolutional layer Pcbr7 are cascaded step by step. The seventh convolutional layer Pcbr7 and the fourth convolutional layer Pcbr4 are concatenated and then cascaded with the eighth convolutional layer Pcbr8. Finally, the output of the eighth convolutional layer Pcbr8 of the three lightweight residual stacks LB is used as one output. The target prediction module includes three convolutional layers: the ninth convolutional layer Pcbs9; the tenth and eleventh convolutional layers Pcbs10 and Pcbs11 are both composed of 1x1 convolutions, batch normalization (BN), and sigmoid; the three-level features extracted by the feature extraction module are respectively input into the three convolutional layers of the target prediction module, and the model prediction information is obtained by concatenation and combination. Four types of target data are input into a lightweight multi-target neural network detection model and trained using deep learning to obtain a converged multi-target detection network model. Then, batch normalization (BN) in the model is merged into the corresponding convolutional convolution, and the floating-point parameters of the model are quantized into 8-bit format, thus completing the solidification of the multi-target detection network model. Step (2) Acquire video frames, normalize the video frame data, and input it into the fixed multi-object detection network model to obtain model inference information. The model inference information includes multiple predicted target boxes, the first of which is the first predicted target box. The format of each predicted target bounding box is as follows: ,in Indicates the category of the target. Indicates the confidence level of the target. This represents the coordinate information of the target; data normalization normalizes the video frame data to [0,1]. After the input frame data img_in is normalized, the output data img_out is generated, so img_out = img_in / 255.0; Step (3) categorize the predicted target boxes from the model prediction information according to their confidence levels. Sort by high to low, then by category Divided into 4 sets, namely the humanoid set Face collection OK gesture collection Palm gesture collection ; A collection of human figures Human bounding boxes with a confidence level less than the set threshold th_h are deleted, and the face set is updated. Face prediction bounding boxes with a confidence level less than the set threshold th_f are deleted, and the OK gesture set is removed. Delete the OK gesture prediction boxes with a confidence level less than the set threshold th_o, and remove the palm gesture set. Palm gesture prediction boxes with a confidence level less than the set threshold th_p are deleted. A collection of human figures Face collection OK gesture collection Palm gesture collection The goals belonging to the same person are considered as that person's goal set. ;No. Personal goal set , Humanoid collection The Middle Personal human-shaped goals Face collection The Middle Personal facial recognition target, A collection of OK gestures The Middle Personal OK gesture goals Palm gesture set The Middle Personal palm gesture goals Represents "or"; target set Allows for empty face or gesture targets; set sequence This refers to the detection results output by the multi-target detection network model; Step (4) inputs the detection results output by the multi-target detection network model into the adaptive discrimination module; the adaptive discrimination module outputs discrimination instructions. If the instruction is to adjust, then step (5) is executed; if the instruction is not to adjust, then step (2) is returned and the next frame is executed; the adaptive discrimination module outputs different discrimination instructions according to different scenarios: If the set sequence There exists a target set in which neither face targets nor gesture targets are empty. Then, in the target set where neither the face target nor the gesture target is empty... In the middle, select the target set with the highest confidence level for the corresponding humanoid target. This serves as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the "ok" gesture, then the adaptive adjustment flag "ada_flag" is set to True and saved, an adjustment command is issued, and the face coordinates corresponding to the adaptive adjustment set are output. Execute step (5); if the gesture target in the adaptive adjustment set is the palm gesture, set the adaptive adjustment flag ada_flag to False and save it, issue a return command, return to step (2), and execute the next frame; If the set sequence There is no set of targets where neither face nor gesture targets are empty. However, there exists a target set where the face target is empty but the gesture target is not empty. Extract the set of targets where the face target is empty and the gesture target is not empty. In the middle, the set of targets with the highest confidence level corresponding to humanoid targets. , as the adaptive adjustment set; if the gesture target in the adaptive adjustment set is the OK gesture, then set the adaptive adjustment flag ada_flag to True and save it, and issue a return command; if the gesture target in the adaptive adjustment set is the Palm gesture, then set the adaptive adjustment flag ada_flag to False and save it, and issue a return command; return to step (2) and execute the next frame; If the set sequence medium target set If both the face target and the gesture target are empty, a return command is issued, returning to step (2) and executing the next frame; Step (5) Adjust the camera so that the target is in the center of the camera frame, as follows: Input adaptive adjustment set corresponding to face coordinates The input video frames are evenly divided horizontally into... If there are 100 bars, then the range corresponding to the middle position is 100. , For the width of the corresponding video frame data, Indicates rounding down; The coordinates of the horizontal center of the face in the image That is, in The first of the corals Each strip position, Rotate the camera and pan the image to center the face in the frame; adjust the number of bars. ;like Then the camera rotates to the left. Then the camera rotates to the right. If so, the camera will not rotate; Step (6) returns to step (2) and executes the next frame.

2. The camera adaptive adjustment method based on multi-target detection as described in claim 1, characterized in that: In step (1), .

Images