Video acquisition method and video acquisition system based on convolutional neural network

Abstract

The present invention discloses a video acquisition method based on a convolutional neural network, comprising: S101: receiving a video acquired by a video acquisition device; S102: extracting key parts from video frames of the video through a convolutional neural network model; S103: adjusting physical parameters of the video acquisition device according to detection information and positioning information of the key parts.

Description

Technical Field

[0001] The present invention generally relates to the field of video image processing, and in particular to a video acquisition method and a video acquisition system based on a convolutional neural network. Background Art

[0002] The purpose of object detection and tracking technology in video content is to detect the object of interest (usually a moving object) in a video sequence and locate it along the time axis, thereby forming the motion trajectory of the object. Object detection and tracking technology has been widely used in different fields, such as intelligent video surveillance, autonomous driving, drone navigation, industrial robots, etc.

[0003] The methods of moving target detection mainly include background difference method, time difference method and optical flow field. These methods are all based on spatial domain detection methods. The first two methods have low computational complexity and good real-time performance, but they both have their application limitations. For example, the background difference method works better when the background is stable, but if there are changes in lighting conditions, weather changes, camera shaking, etc., target detection errors will occur. The time difference method works better when the picture changes little between the relevant frames detected, and changes in lighting conditions will also seriously affect the accuracy of target detection. The optical flow field has high computational complexity and higher detection accuracy than the first two methods, but it also cannot solve the problem of changing lighting conditions.

[0004] Early moving target tracking algorithms mainly searched for candidate targets closest to the target to be tracked in the current frame through template matching. Classic algorithms include Kalman filter, mean shift filter, etc. In recent years, with the maturity of deep learning theory and the improvement of various application frameworks, target tracking algorithms based on deep learning and deep learning and correlation filter, which are essentially classifiers, have become mainstream technologies.

[0005] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0006] In view of at least one defect of the prior art, the present invention provides a video acquisition method based on a convolutional neural network, comprising:

[0007] S101: receiving a video acquired by a video acquisition device;

[0008] S102: extracting key parts from video frames of the video using a convolutional neural network model;

[0009] S103: adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key parts.

[0010] According to one aspect of the present invention, the video acquisition method further comprises: establishing the convolutional neural network model by the following steps:

[0011] Choose a model and training framework;

[0012] Establishing a training set for labeling the key parts;

[0013] The training set is trained using the model and training framework, and the model topology and parameters of the convolutional neural network model are output.

[0014] According to one aspect of the present invention, the model includes one of an R-CNN model, a Fast R-CNN model, a Faster R-CNN model, and a Mask R-CNN model, wherein the video acquisition method further includes:

[0015] A plurality of boxes are selected in a predetermined area of ​​the video frame as alternatives for detecting and locating information of the key parts.

[0016] According to one aspect of the present invention, step S102 comprises: extracting the key parts from multiple video frames of the video through the convolutional neural network model,

[0017] The video acquisition method further comprises: when the same key part is detected in a preset number of consecutive frames, and the coordinate difference and / or size difference of the same key part in the consecutive frames is less than a threshold, determining that the detection of the same key part is consistent,

[0018] Wherein the step S103 comprises: when all the detections of the same key part are consistent, adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key part.

[0019] According to one aspect of the present invention, the video acquisition method further comprises: determining whether different key parts meet the preset position relationship; when different key parts meet the preset position relationship, determining that the detection of the key parts is normal;

[0020] The step S103 includes: when different key parts meet the preset position relationship and all the detections of the same key part are consistent, adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key parts.

[0021] According to one aspect of the present invention, the key part includes the hand of the surgeon and the surgical site of the patient, and the physical parameters include the angle and focal length of the video acquisition device.

[0022] According to one aspect of the present invention, step S103 includes: adjusting the angle of the video acquisition device in the following manner:

[0023] The length of the preset sliding time window is N3 frames, and a series of displacement vectors representing the movement trajectory of a key part are , where 0 ≤ i < N3, and the displacement vector of the key part after the angle of the video acquisition device is adjusted is , The amplitude of satisfies:

[0024] .

[0025] According to one aspect of the present invention, The amplitude of should satisfy

[0026] , where .

[0027] According to one aspect of the present invention, step S103 includes:

[0028] For the calculated , determine whether the key part still falls within a specific field of view area after such adjustment:

[0029] If it falls within the specific field of view area, adjust the angle of the video acquisition device according to the calculated ; otherwise, increase or decrease the value of and recalculate until can fall within the specific field of view area.

[0030] According to one aspect of the present invention, step S103 includes: adjusting the focal length of the video acquisition device according to the proportion of the key part in the video frame.

[0031] The present invention also provides a video acquisition system, including:

[0032] A video acquisition device configured to acquire video in real time;

[0033] An image processing unit, which communicates with the video acquisition device to receive the video acquired by the video acquisition device, and is configured to execute the video acquisition method according to any one of claims 1-10 and adjust the physical parameters of the video acquisition device.

[0034] According to one aspect of the present invention, the image processing unit is further configured to encode the video captured by the video capture device and output a code stream.

[0035] According to one aspect of the present invention, the video acquisition system also includes an ultrasonic imaging device, which communicates with the image processing unit and is configured to acquire ultrasonic images of the patient. The image processing unit is configured to encode the ultrasonic image and output a bit stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0037] Figure 1 Steps for generating a CNN-based key surgical site detection and positioning model

[0038] Figure 2 Camera pan / tilt angle adjustment diagram

[0039] Figure 3 Hardware configuration and functional block diagram of remote ultrasound guidance system based on convolutional neural network

[0040] Figure 4 Workflow diagram of remote ultrasound guidance system based on convolutional neural network

[0041] Figure 5 End-to-cloud integrated remote ultrasound guidance system based on convolutional neural network DETAILED DESCRIPTION

[0042] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0043] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0044] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is lower in level than the second feature.

[0046] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0047] The present invention relates to a remote surgery system and device, which is essentially a video transmission (including video storage in some applications) system, and its application fields include remote surgery teaching, remote surgery guidance, and remote surgery collaboration. By applying a convolutional neural network to the remote surgery system of the present invention, automatic detection and positioning of key surgical sites during surgery are achieved, and the camera's pan / tilt angle, focal length and other physical parameters are automatically adjusted based on the detection and positioning information of the key surgical sites, ensuring that the key surgical sites are not missing in the video image and the image quality is clear; at the same time, using the detection and positioning information of the key surgical sites as retrieval information, structured storage of surgical videos can be achieved, which not only saves storage resources but also facilitates the retrieval and viewing of subsequent content.

[0048] Figure 1 A video acquisition method 100 based on a convolutional neural network according to an embodiment of the present invention is shown and will be described in detail below with reference to the accompanying drawings.

[0049] In step S101: receiving a video acquired by a video acquisition device. The video acquisition device is, for example, a camera. The scene of the video may be a remote surgery scene, which includes image elements such as a surgeon, a patient, surgical instruments, and a surgical site. For convenience, the remote surgery scene will be described below as an example. It is easy for those skilled in the art to understand that the scheme and concept of the present invention can also be applied to video processing in other scenes, which are all within the protection scope of the present invention.

[0050] In step S102: extract key parts from the video frames of the video through the convolutional neural network model. Examples of the key parts include but are not limited to the hands of the surgeon, the surgical site of the patient, etc. The convolutional neural network model can be trained in advance and is specifically used to extract key parts of the video in a specific scene, which will be described in detail below.

[0051] In step S103: adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key parts.

[0052] Taking the video of a remote surgery scene as an example, it is required that key parts such as the surgeon's hands and the patient's surgical site must be included, and it is best to be located in the center of the video screen. Therefore, through the detection and positioning information of key parts, the physical parameters of the video acquisition device, such as shooting angle, focal length and other physical parameters, are adjusted to ensure that key surgical parts are not missing in the video screen and the image quality is clear. For example, when the key parts cannot be detected, it is necessary to adjust the shooting angle of the camera; when the clarity of certain key parts is low, it is necessary to adjust the focal length of the camera to focus on the key parts to obtain a clear display. The detection information includes but is not limited to the name or type of the extracted key parts, or whether certain key parts are extracted. According to one embodiment, the convolutional neural network model identifies the name or type of each key part extracted, and the name or type belongs to the detection information in the present invention.

[0053] According to a preferred embodiment of the present invention, the convolutional neural network model is established in the following manner.

[0054] In step S1, a model and a training framework are selected. The detection and positioning of key parts belong to the category of target detection. Preferably, the model includes one of an R-CNN model, a Fast R-CNN model, a Faster R-CNN model, and a Mask R-CNN model. The training framework can be selected according to different application requirements. For example, when it is necessary to implement pixel-level image segmentation, such as the "AI green screen technology" that can automatically complete the foreground and background segmentation of the picture during video acquisition, the Mask R-CNN framework can be used; the present invention needs to complete the identification and positioning of key parts of the video input signal, and Fast R-CNN can be selected to achieve a better balance between inference speed and inference performance (comprehensive evaluation of multiple indicators such as false positive rate and missed positive rate).

[0055] In step S2, a training set for labeling the key parts is established.

[0056] First, the original training set is selected according to different telesurgery application scenarios. The original training set includes original positive samples and original negative samples. The original positive samples should be selected from pictures of the same surgical content or images extracted from related video content, and the corresponding key parts are marked; the original negative samples can be selected arbitrarily. For the original positive samples, the key surgical parts are cut out and added to the formal training positive sample set; for the original negative samples, different areas can be selected from the same image and added to the formal training negative sample set.

[0057] In step S3, the training set is trained using the model and training framework, and the model topology and parameters of the convolutional neural network model are output. The convolutional neural network model obtained through the above training can be used to identify key parts in the application scenario.

[0058] According to a preferred embodiment of the present invention, the video acquisition method 100 further includes: selecting a plurality of boxes in a predetermined area of ​​the video frame as alternatives for the detection and positioning information of the key parts. For the R-CNN model and similar models, in order to determine the size and position information of the target or key parts to be determined, it is necessary to select some boxes of different sizes, positions, and aspect ratios in a predetermined area of ​​the video image as alternatives for the detection and positioning information of the target to be determined, which are called bounding boxes. Different bounding boxes can be set for different key parts. For example, for the surgeon's hand, a roughly rectangular bounding box can be set; for the patient's head, a roughly square bounding box can be set; for some patients' arms, an elongated bounding box can be set. For the early R-CNN model, the bounding box can be pre-selected. For the Faster R-CNN model, the bounding box is automatically obtained by adding a layer of Region Proposal Network (RPN) to the R-CNN model. Specifically for the detection and positioning of key surgical parts, it is necessary to choose how to obtain the bounding box according to actual application requirements. If there is only one key part in the application and its size and position in the video picture are strictly restricted, a small number of bounding boxes can be preset; if there are more than one key parts in the application and there are no strict restrictions on their size and position in the video picture, it is preferred to use RPN to automatically obtain a set of bounding boxes.

[0059] According to one embodiment of the present invention, in step S102, key parts are extracted from a single frame of the video through the convolutional neural network model, and the physical parameters of the video acquisition device are adjusted according to the detection information and positioning information of the key parts.

[0060] In addition, preferably, taking remote surgery video as an example, a trained neural network model (such as an R-CNN model) is applied to all or some frames of the remote surgery input video signal to infer information about key surgical sites, including target detection information and positioning information. Preferably, when the same key site is detected in a preset number of consecutive frames (for example, two or more frames), and the coordinate difference and / or size difference of the same key site in the consecutive frames is less than a threshold, it is judged that the detection of the same key site is consistent. For example, by extracting key sites from a single frame or multiple frames of the video using the convolutional neural network model, the category of the detected target can be annotated at the same time. For example, for a remote surgery system with K key surgical sites, it can be labeled as Object. k , k=1,…,K. The positioning information of each identified key surgical site can be described by a rectangular box (x, y, w, h), where x and y are the coordinates of the upper left corner of the rectangle, and w and h are the width and height of the rectangle respectively. Therefore, the complete information descriptor of the key surgical site is Object k (x k ,y k , w k ,h k ). According to an embodiment of the present invention, the neural network model does not have to be applied to every frame of the input video signal. The neural network model can be extracted and applied according to the specific situation. For example, the neural network model is applied to extract key surgical site information every (N1-1) frames (N1 is a system preset value). Without loss of generality, in the subsequent description of the present invention, it is assumed that the R-CNN model or the above-mentioned other models are applied to every frame of the input video signal.

[0061] In the initial stage of the remote surgery system receiving the input video signal, it is necessary to detect the key surgical site. Accurate detection of the key surgical site is very important for subsequent tracking and positioning. Therefore, it is generally necessary to apply the R-CNN model or other models mentioned above to multiple frames to screen all the extracted key surgical site information. The screening method is as follows.

[0062] For a key surgical site Object k , at least N2 consecutive frames of the video image to be detected need to be detected. k Only when the detection information is consistent can the target be correctly detected and the tracking and positioning of the key surgical site be started. N2 is a system preset value, for example, greater than or equal to 2. "Detection information is consistent" includes that the coordinate difference and / or size difference of the same key site in multiple consecutive frames is less than the threshold, that is, one or both of the following two conditions:

[0063] Detected in the video image where N2 consecutive frames are detected Complete information Should meet:

[0064]

[0065] Where d, , All are system preset values, i, j satisfy The first condition defines the coordinate difference, and the second condition defines the size difference. Other formulas can also be used to define the coordinate difference and size difference.

[0066] The step S103 includes: when all the detections of the same key part are consistent, adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key part. If the detections of the same key part are inconsistent, the physical parameters of the video acquisition device are not adjusted.

[0067] According to a preferred embodiment of the present invention, the video acquisition method further includes: determining whether different key parts meet the preset position relationship; when different key parts meet the preset position relationship, determining that the detection of the key parts is normal. When it is determined that it does not meet the preset position relationship, it is considered to be a false detection, and the detection result is discarded. The following example is explained in detail.

[0068] If all or part of the K key surgical sites in the system have relative positional relationships, such as and Must appear at the same time and have overlapping parts, then during the screening process, you need to confirm whether the following conditions are met:

[0069]

[0070] Satisfying (Eq.2) and To be determined to be correctly detected.

[0071] The step S103 includes: when different key parts meet the preset position relationship and all the detections of the same key parts are consistent, adjusting the physical parameters of the video acquisition device, such as the angle and focal length of the video acquisition device, according to the detection information and positioning information of the key parts.

[0072] After completing the detection of key surgical sites, the key surgical sites can be detected and located on the input video signal, that is, the R-CNN model is applied to each frame of the video and the output .

[0073] According to a preferred embodiment of the present invention, the step S103 includes: adjusting the angle of the video acquisition device in the following manner.

[0074] The length of a sliding time window is preset to be N3 frames, and the i-th frame extracted in the time window is The complete information is N3 is a system preset value, for example, greater than or equal to 2.

[0075] This time window represents A series of displacement vectors of the motion trajectory like Figure 2 As shown (N3=3), after the gimbal angle is adjusted The displacement vector is ,but The direction should be Figure 2 As shown by the dashed arrow ( Figure 2 As shown); in order to avoid "over-adjustment", the gimbal angle should be adjusted in the direction The amplitude should satisfy:

[0076]

[0077] In practical applications, a coefficient less than 1 can be chosen ,make

[0078]

[0079] At the same time, an area can be pre-demarcated in the video screen according to the actual application needs. The "effective area" is calculated each time After that, check whether the gimbal angle is adjusted in this way. Is it still within the "valid area": ​​If yes, the adjustment is valid, and the system sends a command to the camera to adjust the PTZ angle; if not, increase or decrease Recalculate the value of ,until Can fall within the "valid area".

[0080] In addition, the focal length of the video acquisition device can be adjusted according to the proportion of the key parts in the video frame. The focal length of the camera can be automatically adjusted based on the detection and positioning information of the key surgical parts. The purpose of automatically adjusting the focal length of the camera is to ensure that the size of the key surgical parts presented in the video screen is appropriate (if it is too small, the details cannot be presented clearly, and if it is too large, part of it may fall outside the video screen). The optional methods are as follows:

[0081] The length of a sliding time window is preset to be N4 frames, and the i-th frame extracted in the time window is The complete information is N4 is a system preset value, for example, greater than or equal to 2. If the following equation is satisfied for all i:

[0082]

[0083] The system needs to send a command to the camera to adjust the zoom to zoom in and enlarge the image.

[0084] On the contrary, if the following equation is satisfied for all i:

[0085]

[0086] The system needs to send a command to the camera to adjust the zoom to zoom out the image. Where S is the area of ​​the video image, for example, the area calculated in pixels. and are all system preset values ​​greater than zero and less than 1 and .

[0087] The present invention also relates to a video acquisition system, comprising: a video acquisition device configured to acquire video in real time; an image processing unit (a video intelligent terminal, such as Figure 3 and Figure 5 As shown), the image processing unit communicates with the video acquisition device to receive the video acquired by the video acquisition device, and is configured to execute the video acquisition method as described above and adjust the physical parameters of the video acquisition device.

[0088] According to one aspect of the present invention, the image processing unit is further configured to encode the video captured by the video acquisition device and output a code stream. According to one embodiment of the present invention, the video acquisition system further includes an ultrasound imaging device, the ultrasound imaging device communicates with the image processing unit and is configured to capture ultrasound images of the patient, and the image processing unit is configured to encode the ultrasound image and output a code stream.

[0089] [Example 1]

[0090] Figure 3 and Figure 4 A system according to the first embodiment of the present invention is shown, which is a remote ultrasound guidance system based on a convolutional neural network. Figure 3 The hardware configuration and functional block diagram of the system are shown below. Figure 4The flowchart of the system is shown below. The system is a remote ultrasound guidance system based on convolutional neural networks. The remote ultrasound guidance system is an important scenario in telemedicine and computer-assisted distance learning. In the remote ultrasound guidance system, it is necessary to clearly present the ultrasound probe, the body examination part and the doctor's examination technique in the video screen. Therefore, two video input signals are configured in the system, one video is used to shoot the ultrasound examination technique in real time through a pan-tilt camera; the other video is connected to the ultrasound image output. Both videos are encoded and transmitted in real time. The system can implement the video acquisition method 100 based on convolutional neural networks as described above.

[0091] Among them, the pan-tilt camera shoots the ultrasound examination technique in real time. Later, the key surgical parts in the video need to be detected and tracked in real time, so it is preferred that the pan-tilt camera selects a high-definition non-compressed format camera device.

[0092] In addition, the video intelligent terminal is the core device of the system, which receives the output signals of the PTZ camera and the ultrasonic imaging device, processes and transmits them in real time. Its functions include:

[0093] i) extracting detection and positioning information of key surgical parts from the output signal of the pan-tilt camera in real time, and issuing a command to adjust the pan-tilt angle and / or camera focal length to the pan-tilt camera by performing real-time analysis on the detection and positioning information of key surgical parts;

[0094] ii) Perform real-time high-definition video encoding on the output signal of the PTZ camera and output the bitstream signal to the terminal receiving device specified by the application;

[0095] iii) Perform real-time high-definition video encoding on the output signal of the ultrasound imaging device and output the bit stream signal to the terminal receiving device specified by the application.

[0096] The workflow of the remote ultrasound guidance system based on convolutional neural network is as follows Figure 4 shown.

[0097] First, according to the above embodiment of the present invention, offline model training is performed according to the steps of generating a key surgical site detection and positioning model based on a convolutional neural network, and videos of ultrasound examinations and ultrasound surgeries are selected as original training sets, and human hands and ultrasound probes are selected as key surgical sites. When performing model training, it is preferred to perform R-CNN model training on hardware with high parallel computing capabilities, such as a computer cluster equipped with multiple GPUs.

[0098] The trained R-CNN model is then used to process the video input signal of the PTZ camera to detect human hands and ultrasonic probes; based on the detection results, the relevant parameters of the PTZ camera are adjusted, including the angle and / or focal length of the camera.

[0099] Next, the video input signal of the PTZ camera is encoded in real time with high quality and the code stream is output.

[0100] In addition, for the ultrasound images output by the ultrasound imaging device, real-time high-definition video encoding can also be performed and the code stream can be output (not in Figure 4 (displayed in the figure).

[0101] [Example 2]

[0102] Embodiment 2 is a special case of end-cloud integration of embodiment 1, which is applicable to a remote ultrasound guidance system in a high-bandwidth, low-latency mobile communication network environment. Figure 5 shown.

[0103] exist Figure 4 On this basis, the video communication terminal in the remote ultrasound guidance system is upgraded to a cloud video intelligent communication terminal, and the output code stream is transmitted to the cloud platform via a high-bandwidth and low-latency mobile communication network. The cloud platform distributes the code stream to different terminal users via different transmission networks, including mobile video communication terminals, mobile phones, computers, etc.

[0104] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video acquisition method based on convolutional neural network, comprising: S101: receiving a video acquired by a video acquisition device; S102: extracting key parts from video frames of the video using a convolutional neural network model; S103: adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key parts; The step S102 comprises: extracting the key parts from multiple frames of the video using the convolutional neural network model, The video acquisition method further comprises: when the same key part is detected in a preset number of consecutive frames, and the coordinate difference and / or size difference of the same key part in the consecutive frames is less than a threshold, determining that the detection of the same key part is consistent, Wherein the step S103 comprises: when all the detections of the same key part are consistent, adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key part.

2. The video acquisition method according to claim 1, further comprising: The convolutional neural network model is established by following the steps below: Choose a model and training framework; Establishing a training set for labeling the key parts; The training set is trained using the model and training framework, and the model topology and parameters of the convolutional neural network model are output.

3. The video acquisition method according to claim 2, wherein the model comprises one of an R-CNN model, a Fast R-CNN model, a Faster R-CNN model, and a Mask R-CNN model, wherein the video acquisition method further comprises: A plurality of boxes are selected in a predetermined area of ​​the video frame as alternatives for detecting and locating information of the key parts.

4. The video acquisition method according to any one of claims 1 to 3, further comprising: Determine whether different key parts meet the preset position relationship; When different key parts meet the preset position relationship, it is determined that the detection of the key parts is normal; The step S103 includes: when different key parts meet the preset position relationship and all the detections of the same key part are consistent, adjusting the physical parameters of the video acquisition device according to the detection information and positioning information of the key parts.

5. The video acquisition method as claimed in claim 4, wherein the key parts include the hands of the surgeon and the surgical site of the patient, and the physical parameters include the angle and focal length of the video acquisition device.

6. The video acquisition method according to claim 5, wherein the step S103 comprises: The angle of the video acquisition device is adjusted by: The length of the preset sliding time window is N3 frames, and a series of displacement vectors representing the motion trajectory of a key part are , where 0 ≤ i < N3, and the displacement vector of the key part after the angle adjustment of the video acquisition device is , The amplitude of satisfies: 。 7. The video acquisition method according to claim 6, wherein The amplitude should satisfy ,in .

8. The video acquisition method according to claim 7, wherein the step S103 comprises: For the calculated , determine whether the key part still falls within the specific field of view after such adjustment: If it falls within a specific field of view, according to the calculated Adjust the angle of the video acquisition device; otherwise, increase or decrease Recalculate the value of ,until can fall within the specific field of view area.

9. The video acquisition method according to claim 5, wherein the step S103 comprises: The focal length of the video acquisition device is adjusted according to the proportion of the key part in the video frame.

10. A video acquisition system, comprising: A video acquisition device configured to acquire video in real time; An image processing unit, which communicates with the video acquisition device to receive the video acquired by the video acquisition device, and is configured to execute the video acquisition method as described in any one of claims 1 to 9, and adjust the physical parameters of the video acquisition device.

11. The video acquisition system according to claim 10, wherein the image processing unit is further configured to encode the video acquired by the video acquisition device and output a code stream.

12. The video acquisition system as described in claim 10 or 11 further includes an ultrasonic imaging device, which communicates with the image processing unit and is configured to acquire ultrasonic images of the patient, and the image processing unit is configured to encode the ultrasonic image and output a bit stream.

Images