Camera control device and camera

Abstract

The embodiment of the invention provides a camera control device and a camera. The control device comprises a main processor, a coprocessor and an AI chip, when the control device is used for detecting a shooting main body and automatically determining a focusing area, an image preprocessing and target detection network can be deployed on the AI chip which is good at matrix correlation operation, and an image normalization processing process is deployed on the coprocessor which is good at floating point operation. The post-processing process of the output result of the target detection network and the focusing control process of the camera are deployed on the main processing of skilled logic operation, the focusing area is automatically determined through the cooperation of a plurality of processors, and each processor executes the skilled task, so that the processing speed can be greatly improved, and the processing efficiency is improved. And the real-time requirement of the live broadcast scene is met.

Description

Camera control device and camera Technical Field

[0001] This specification relates to the field of photographing technology, and in particular to a camera control device and a camera. Background Art

[0002] When taking pictures with a camera, due to the different distances between objects in the scene and the camera and the influence of the depth of field of the camera lens, it is impossible for all objects in the entire field of view to be clear. Therefore, it is necessary to select a small area of ​​interest to the user (ROI area) as the focus area and focus based on the focus area so that the objects in the focus area are clearly imaged.

[0003] Currently, there are two methods for determining the focus area. One is for the user to manually determine the focus area within the image, which is relatively cumbersome. The other method is to automatically detect the subject from the image and use the image area where the subject is located as the focus area. This method requires no manual user operation and is more intelligent. However, when implementing automatic subject detection and focus, the current technology has a slow processing speed, with a frame rate generally not exceeding 30 frames per second, which cannot meet the needs of some scenes with high real-time requirements.

[0004] Summary of the Invention

[0005] Based on this, this specification provides a camera control device and a camera.

[0006] According to a first aspect of an embodiment of this specification, a camera control device is provided, the control device comprising a main processor, a coprocessor, and an AI chip, wherein a pre-trained object detection network is deployed in the AI ​​chip;

[0007] The AI ​​chip is used to preprocess the image captured by the camera to obtain a preprocessed image that meets the format requirements of the target detection network;

[0008] The coprocessor is used to perform normalization processing on the preprocessed image to obtain normalized image data;

[0009] The AI ​​chip is used to input the normalized image data into the object detection network and obtain an output result of the object detection network, wherein the output result is used to determine the position of the photographed subject in the image;

[0010] The main processor is used to determine the position of the shooting subject in the image according to the output result, and determine a focus area based on the position, so as to control the focus of the camera based on the focus area.

[0011] According to a second aspect of an embodiment of this specification, a camera is provided, comprising an image sensor, a lens, and the control device mentioned in the first aspect above, wherein the image sensor is used to capture an image and send it to the control device, and the control device is used to control the focus of the lens based on the image.

[0012] Applying the embodiment scheme of this specification, considering that some existing embedded hardware processing platforms usually include multiple processors such as a main processor, a coprocessor, and an AI chip, when using this type of embedded hardware processing platform to detect the shooting subject and automatically determine the focus area, the image preprocessing and target detection network can be deployed on an AI chip that excels in matrix-related operations, the image normalization processing process can be deployed on a coprocessor that excels in floating-point operations, and the post-processing process of the output results of the target detection network and the focus control process of the camera can be deployed on the main processor that excels in logical operations. By cooperating with multiple processors to achieve automatic determination of the focus area, and each processor performs the task it is good at, the processing speed can be greatly improved to meet the real-time requirements of the live broadcast scene.

[0013] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0015] FIG1 is a schematic diagram of an embedded hardware processing platform of the prior art according to an embodiment of the present specification.

[0016] FIG. 2 is a schematic diagram of a camera control device according to an embodiment of the present disclosure.

[0017] FIG3 is a schematic diagram of an embodiment of the present specification in which the upper body of a human body is used as a photographic subject.

[0018] FIG. 4 is a schematic diagram of determining a focus object from multiple shooting subjects according to an embodiment of this specification.

[0019] FIG5 is a schematic diagram of an object detection network outputting multiple candidate boxes according to an embodiment of this specification.

[0020] FIG6 is a schematic diagram of various processing steps for determining a focus area deployed on an embedded hardware processing platform of a live camera according to an embodiment of the present specification.

[0021] FIG. 7 is a schematic diagram of the structure of a camera according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0023] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0025] When taking photos with a camera, due to the varying distances between objects in the scene and the camera and the depth of field of the camera lens, it's impossible for all objects within the entire field of view to be in focus. Therefore, a small area of ​​interest (ROI) needs to be selected as the focus area. Focusing is then performed based on this focus area to ensure that the objects within the focus area are clearly imaged. Currently, there are two methods for determining the focus area. One method involves the user manually operating the image, such as by clicking or framing the subject, and then determining the area within the subject as the focus area. This method requires manual operation for each shot, which is cumbersome and not intelligent enough, and cannot meet the needs of some scenarios that require continuous focus (such as tracking shots). Another method is to automatically identify the subject from the image and select the image area containing the subject as the focus area. For example, if the subject is a face, the face's position can be automatically identified from the image before focusing, and the area within the face can then be designated as the focus area.

[0026] To address the need for automatic focus area identification, some manufacturers have developed embedded hardware processing platforms for camera focus control. As shown in FIG1 , these hardware processing platforms 10 typically include a detection chip 11 specifically for detecting the subject and a CPU 12. Of course, other processors can also be configured based on actual needs. The detection chip 11 detects the subject's position in the image and then sends this position information to the CPU 12, which then determines the focus area based on this position information and controls the camera focus. Since subject detection typically requires a neural network, which involves a significant amount of computation, the processing power of the detection chip 11 in the embedded processing platform is generally limited. Consequently, the process of detecting the subject in the image and determining the focus area is time-consuming, resulting in a very limited frame rate, typically no more than 30 frames per second. However, for scenarios requiring high real-time performance, such as live broadcasts that require real-time tracking of the subject, high processing speed requirements are required, and current embedded hardware processing platforms are unable to meet these requirements.

[0027] Considering that some existing embedded hardware processing platforms for cameras often include multiple processors, for example, some hardware processing platforms include CPUs, DSP chips, GPUs, or other coprocessors, each of these processors excels at different types of operations. Therefore, when automatically detecting the subject from an image to determine the focus area, the task of detecting the subject can be split into multiple subtasks. Then, based on the characteristics of the data processing involved in each subtask, each subtask is assigned to a processor that excels at processing that subtask. In this way, the processing speed of detecting the subject from an image can be greatly improved, thereby meeting the autofocus requirements in scenarios with high real-time requirements, such as live broadcasts.

[0028] Based on this, an embodiment of the present disclosure provides a camera control device. As shown in FIG2 , the control device 20 can be used to control the camera autofocus. In some embodiments, the control device 20 can be an embedded hardware processing platform provided on the camera. The control device 20 includes a main processor 21, a coprocessor 22, and an AI chip 23. The main processor 21 can be any processor that is good at performing logical operations, such as a CPU or a processor with similar characteristics. The coprocessor 22 can be any processor that is good at performing floating-point operations. The AI ​​chip 23 can be a processor that is good at processing images or performing some matrix-related operations.

[0029] In some embodiments, the main processor 21 may be a CPU, the coprocessor 22 may be a Neon processor, and the AI ​​chip may be a DSP chip.

[0030] Generally speaking, before officially capturing an image, a camera can capture some preview images. By analyzing these preview images, it can determine some shooting parameters, such as aperture size, shutter speed, and focus area. Therefore, before officially capturing an image, the image sensor can capture a preview image of the current scene. The position of the subject can then be detected from the preview image to determine the focus area based on this position. When capturing the next image frame, the camera lens focus can be controlled based on the currently determined focus area.

[0031] In order to automatically detect the subject from the image and focus on it, a target detection network can be pre-trained using labeled sample images and then used to detect the subject. Since the target detection network primarily performs matrix-related operations such as convolution when determining the subject's position in the image, the target detection network can be deployed on an AI chip 23 that excels at such operations to increase the processing speed of such operations.

[0032] Typically, the target detection network has certain requirements for the format and resolution of the input image. Therefore, the AI ​​chip 23 can be used to preprocess the image captured by the camera to obtain a preprocessed image that meets the format requirements of the target detection network. For example, the image output by the camera is usually in NV12 format, while the image processed by the target detection network is usually in RGB format. Therefore, the NV12 format image can be converted into RGB format before inputting it into the target detection network for detection.

[0033] In addition, before inputting the image into the target detection network, it is usually necessary to normalize the image data. For example, the pixel values ​​of the R, G, and B channels of the image are usually between 0 and 255. In order to facilitate the calculation of the target detection network and improve the processing speed, the pixel values ​​of the three channels can be normalized first, for example, they can be mapped to between 0 and 1. The normalization process involves a large number of floating-point operations. To improve the processing speed of this step, the coprocessor 22, which is good at floating-point operations, can be used to normalize the pre-processed image to obtain normalized image data.

[0034] After obtaining the normalized image data, the AI ​​chip 23 can input the normalized image data into a pre-trained target detection network, and the target detection network performs a series of processing on the normalized image data, such as extracting features from the image, classifying objects in the image based on the extracted features, etc., to obtain an output result. Among them, the output result can be various types of information that can be used to determine the position of the photographed subject in the image. The output results are different for different types of target detection networks. For example, if the target detection network is an SSD target detection network (Single Shot MultiBox Detector), its output result can be the position information of multiple candidate boxes determined from the image, and the category score corresponding to each candidate box. The category score is used to evaluate the probability that the area corresponding to the candidate box is a preset one or more objects. Of course, for other types of target detection networks, the output result can be other information.

[0035] After obtaining the output results of the target detection network, the output results may need to be further post-processed to determine the position of the subject in the image. Generally, the operations involved in post-processing, as well as determining the focus area based on the position of the subject in the image and controlling the camera focus based on the focus area, mainly involve some logical operations. Therefore, these steps can be performed by the main processor 21 that is good at performing logical operations. The main processor 21 can post-process the output results of the target detection network. For example, if the output results include multiple candidate frames, the main processor 21 can perform NMS (NMS: Non Maximum Suppression) processing on these multiple candidate frames to remove some candidate frames with more overlapping areas among the multiple candidate frames. The main processor 21 can also normalize the category scores of each candidate frame to obtain the probability that each candidate frame belongs to each object category, and then determine the detection frame of the subject from the candidate frames based on the probability, and use the position of the detection frame as the position of the subject.

[0036] After determining the position of the subject in the image, the main processor 21 can determine a focus area in the image based on the position of the subject, so as to control the camera focus according to the focus area to ensure a clear image of the subject. For example, a phase focus method can be used to control the camera to automatically focus on the focus area. Of course, other autofocus methods can also be used.

[0037] Considering that some existing embedded hardware processing platforms usually include multiple processors such as a main processor, a coprocessor, and an AI chip, when using such embedded hardware processing platforms to detect the shooting subject and automatically determine the focus area, the image preprocessing and target detection network can be deployed on an AI chip that excels in matrix-related operations, the image normalization processing process can be deployed on a coprocessor that excels in floating-point operations, and the post-processing process of the output results of the target detection network and the focus control process of the camera can be deployed on the main processor that excels in logical operations. By working together with multiple processors to automatically determine the focus area, and each processor performs the task it is good at, the processing speed can be greatly improved to meet the real-time requirements of live broadcast scenarios.

[0038] In most photography scenarios, the user's focus is on the people in the scene. For example, in live broadcasts or other tracking photography scenarios, faces are often the subject of the shot. Current technology typically uses a face detection algorithm to automatically detect faces in the image, then controls the camera focus based on the face's location to ensure a clear face. However, while face detection can generally accurately detect faces from the front, it struggles with profiles or scenes with faces facing away from the camera, often leading to missed detections. This results in the profile or head being out of focus, resulting in unclear images of these faces. To address this issue, in some embodiments, when the user wants to capture a person, the entire upper body can be used as the subject. Compared to detecting faces, detecting the upper body is more accurate and stable, reducing the problem of unclear images due to missed detections. Furthermore, for tracking photography scenarios, detecting the upper body can also improve tracking of the target person.

[0039] For scenes where the subject is the upper body of the human body, when training the target detection network, as shown in Figure 3, a large number of sample images marked with detection frames corresponding to the upper body of the human body can be obtained, and these sample images can be used to train the target detection network.

[0040] Furthermore, in scenarios where the subject is a face, if there are multiple faces within the viewing angle, current technology can only detect the positions of each face, but cannot determine the specific subject to be focused on. Therefore, in some embodiments, if the image includes multiple subjects, the main processor 21 is further configured to, after determining the positions of each subject in the image, determine the distance between each subject and the center of the image, as well as the depth information of each subject based on the position of each subject. The main processor 21 then determines the focus subject from the multiple subjects based on at least one of the two pieces of information, and uses the area containing the focus subject as the focus area. For example, the subject closest to the center of the image (i.e., located at the center of the frame) can be used as the focus subject, or the subject with the smallest depth information (i.e., the foreground object in the image) can be used as the focus subject. The depth information represents the distance between the subject and the camera. In some scenarios, the camera may include a rangefinder, and the distance between the subject and the camera can be measured based on the rangefinder. In some scenarios, the distance between the subject and the camera can also be determined based on the parallax of images captured by the camera at different angles.

[0041] In some embodiments, as shown in FIG4 , considering that the object that is usually located at the center of the picture and in the foreground of the picture is the object that the user is most concerned about, the subject with the smallest distance from the center of the image and the smallest depth information among multiple subjects can be used as the focus object.

[0042] Typically, target detection networks have certain requirements for the resolution of input images. For example, the SSD (Single Shot MultiBox Detector) target detection network typically requires the input image to be a color image with a resolution of 300×300. However, the resolution of images captured by cameras is often higher. Therefore, in some embodiments, when pre-processing the image captured by the camera, the AI ​​chip 23 may convert the resolution of the image to a resolution that matches the target detection network. For example, the image captured by the camera may be downsampled first and converted into a color image with a resolution of 300×300.

[0043] In addition, the target detection network also has certain requirements for the format of the input image. For example, in general, the target detection network requires the input image to be in RGB format, while the image captured by the camera is usually in NV12 format. Therefore, in some embodiments, when pre-processing the image, the AI ​​chip 23 can convert the image format into a format that meets the requirements of the target detection network. For example, the image in NV12 format can be first converted into an image in RGBA8888 format, and then converted into an image in RGB format.

[0044] In some embodiments, as shown in FIG5 , the target detection network may be an SSD (Single Shot MultiBox Detector) target detection network. The output of the target detection network may be the position information of multiple candidate boxes determined from the image, as well as the category score corresponding to each candidate box. The category score is used to evaluate the probability that the area corresponding to the candidate box is one or more objects preset for the subject. When determining the position of the subject in the image based on the output result, the main processor 21 may first post-process the output result. For example, linear regression correction may be performed on the position and size of each candidate box to correct the size and position of each candidate box. In addition, since there are multiple candidate boxes corresponding to the same object in the image, these candidate boxes may have many overlapping areas. In order to eliminate some candidate boxes with large overlapping areas, non-maximum suppression (NMS) may be further performed on the candidate boxes. For example, for multiple candidate boxes corresponding to an object A, the candidate box with the highest score may be selected from the multiple candidate boxes as a reference. The overlapping area between the remaining candidate boxes and the candidate box is then determined. If the overlapping area is greater than a preset threshold, the remaining candidate boxes are removed. After non-maximum suppression, a small number of candidate object frames remain. These frames' category scores are then normalized to determine the probability that each frame belongs to a specific object category. Finally, based on these probabilities, the subject's detection frame is determined from the candidate object frames, thereby determining the subject's location. The image area corresponding to the detection frame can then be used as the focus area to control the camera lens' focus.

[0045] In some embodiments, the control device can be an embedded hardware processing platform within the camera. Given the limited processing performance of embedded hardware processing platforms, to improve their processing efficiency and reduce the computational and storage resources consumed by the target detection network, lightweight neural networks can be selected when selecting target detection networks. For example, a lightweight MobileNet network can be used as the backbone network of the target detection network to better meet the requirements of the embedded hardware processing platform.

[0046] In some embodiments, the camera may be a live broadcast camera, and the control device may be an embedded hardware processing platform provided in the camera. Since live broadcast scenarios have high real-time requirements, and the performance of general embedded hardware processing platforms is limited, it is difficult to meet real-time requirements. However, when detecting the subject to be photographed and determining the focus area, the disclosed embodiment can make full use of the performance of each processor in the hardware processing platform, split the entire task, and deploy the split tasks on processors that are good at processing such tasks. In this way, a higher processing speed can be achieved for scenarios with poor computing performance of the hardware processing platform. That is, it can simultaneously meet the requirements of poor performance of the embedded hardware processing platform and high real-time requirements of the live broadcast camera.

[0047] In addition, an embodiment of the present disclosure provides a camera, as shown in Figure 7, which includes an image sensor, a lens and the control device mentioned in any of the above embodiments. After the image sensor captures the image, it can first send it to the control device, which detects the shooting subject in the image, determines the focus area based on the position of the shooting subject, and controls the focus of the camera lens.

[0048] The specific structure of the control device, as well as the specific process of determining the shooting subject, determining the focus area based on the shooting subject, and focusing the lens can be referred to the description in the above embodiments and will not be repeated here.

[0049] In order to further explain the control device and camera provided by the embodiments of the present disclosure, a specific embodiment is used for explanation below.

[0050] Live cameras are typically used to capture live video of a live broadcast, which is then streamed to a live broadcast server and distributed to viewer clients. Typically, during a live broadcast, the focus is on people, and therefore it is desirable to capture them clearly. To automatically focus on people in each frame, the person can be detected from the image, and then the area where the person is located can be used as the focus area. Considering that face detection cannot accurately detect faces in profile or with their backs to the camera, in this embodiment, the upper body of the human body is used as the subject of the shot. By detecting the upper body of the human body from the image, stable focus can be achieved.

[0051] Furthermore, because live streaming scenarios require high real-time performance, it is desirable to detect the upper body area as quickly as possible to achieve a high processing frame rate. To meet this real-time requirement, this example deploys an object detection network for upper body detection on the embedded hardware processing platform of the live streaming camera. The following improvements are made to the selection and deployment of the object detection network:

[0052] 1. In this embodiment, an SSD object detection network is used to detect the upper body area of ​​the human body. Considering the limited computing and storage resources of the embedded processing platform, Mobilenet is selected as the backbone network. Since Mobilenet is a lightweight network with low requirements for computing and storage resources, it is easy to deploy on the embedded hardware processing platform.

[0053] 2. As shown in Figure 6, the embedded hardware processing platform includes a CPU, a Neon coprocessor, and a DSP chip. The entire process of using the target detection network to detect the upper body region of the human body and automatically focusing based on the detected upper body region can be roughly divided into the following stages: pre-processing of the image input to the target detection network (pre-processing primarily includes converting the NV12 format image to an RGBA8888 format image, and then converting it to an RGB format image), image normalization, the target detection network's inference process (the inference process primarily includes outputting the location of the candidate boxes and the corresponding category scores of the candidate boxes), post-processing of the target detection network's output results (the post-processing process primarily includes linear regression correction of the candidate boxes, non-maximum suppression processing, normalization of the category scores, obtaining the probability that the candidate boxes belong to each object category, and determining the detection box for the upper body region), and focus control (focusing the camera lens based on the location of the detection box for the upper body region). Considering that pre-processing and inference involve numerous matrix-related operations, these two steps are deployed on a DSP chip, which excels in matrix operations. Normalization involves numerous floating-point operations, so this step is deployed on a Neon coprocessor, which excels in floating-point operations. Post-processing and focus control involve numerous logical operations, so they are deployed on a CPU, which also excels in logical operations. This deployment significantly improves the processing speed of upper body detection, meeting the real-time autofocus requirements of live broadcast scenarios.

[0054] In addition, if multiple upper body areas are detected, the upper body area closest to the center of the picture and with the smallest depth information can be used as the focus area, that is, the focus object can be automatically determined from multiple human bodies.

[0055] The live broadcast camera provided in this embodiment can quickly and stably determine the focus area, improve the automation performance of the live broadcast camera focus, and the quality of the images collected by the live broadcast camera in actual usage scenarios, which can meet the real-time requirements of the live broadcast camera.

[0056] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of disclosure of this specification.

[0057] Those skilled in the art will readily appreciate other implementations of the embodiments of the present invention after considering and practicing the description disclosed herein. The embodiments of the present invention are intended to cover any variations, uses, or adaptations of the embodiments of the present invention that follow the general principles of the embodiments of the present invention and include common knowledge or customary techniques in the art not disclosed in the embodiments of the present invention. The description and examples are to be considered as exemplary only, and the true scope and spirit of the embodiments of the present invention are indicated by the following claims.

[0058] It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present invention is limited only by the appended claims.

[0059] The above description is only a preferred embodiment of the embodiments of this specification and is not intended to limit the embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of this specification should be included in the scope of protection of the embodiments of this specification.

Claims

1. A camera control device, characterized in that: The control device includes a main processor, a coprocessor and an AI chip, wherein a pre-trained target detection network is deployed in the AI ​​chip; The AI ​​chip is used to preprocess the image captured by the camera to obtain a preprocessed image that meets the format requirements of the target detection network; The coprocessor is used to perform normalization processing on the preprocessed image to obtain normalized image data; The AI ​​chip is used to input the normalized image data into the object detection network and obtain an output result of the object detection network, wherein the output result is used to determine the position of the photographed subject in the image; The main processor is used to determine the position of the shooting subject in the image according to the output result, and determine a focus area based on the position, so as to control the focus of the camera based on the focus area.

2. The control device according to claim 1, characterized in that The photographed subject includes the upper body area of ​​the human body.

3. The control device according to claim 1, characterized in that The image includes multiple shooting subjects, and the main processor is further used to determine a focus object from the multiple shooting subjects based on the distance between each shooting subject and the center of the image and / or the depth information of each shooting subject, and use the area where the focus object is located as the focus area.

4. The control device according to claim 3, characterized in that The focused object is the photographing subject with the smallest distance and the smallest depth information.

5. The control device according to claim 1, characterized in that The AI ​​chip is used to pre-process the images captured by the camera, including: The AI ​​chip is used to convert the resolution of the image into a resolution that matches the object detection network; and / or The AI ​​chip is used to convert the format of the image into a format that meets the requirements of the target detection network.

6. The control device according to claim 1, characterized in that The object detection network includes a Single Shot MultiBox Detector network, the output result includes the positions of multiple candidate boxes determined from the image, and a category score for each candidate box, the category score being used to indicate the probability that each candidate box belongs to one or more preset object categories, and the main processor is used to determine the position of the photographed subject in the image based on the output result, including: The main processing is used to perform linear regression correction processing on the position and size of each candidate frame, and perform non-maximum suppression processing on the multiple candidate frames after linear regression processing to obtain a target candidate frame; the category score of the target candidate frame is normalized to obtain the probability that the target candidate frame belongs to one or more preset object categories, and based on the probability, a detection frame of the shooting subject is selected from the target candidate frame, and the position of the detection frame is used as the position of the shooting subject.

7. The control device according to claim 1, characterized in that The main processor is a CPU, the coprocessor is a Neon coprocessor, and the AI ​​chip is a DSP chip.

8. The control device according to claim 1, characterized in that The backbone network of the target detection network is the Mobilenet network.

9. The control device according to claim 1, characterized in that The camera is a live broadcast camera, and the control device is an embedded hardware processing platform in the live broadcast camera.

10. A camera, characterized in that: The camera includes an image sensor, a lens, and the control device according to any one of claims 1 to 9. The image is collected by the image sensor, and the control device is used to control the focus of the lens.

Images