Target detection and tracking method, electronic equipment and storage medium

Abstract

The invention discloses a target detection and tracking method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected image containing personnel in a target scene by using a camera; inputting the to-be-detected image into a target detection model comprising a first detection head and a second detection head for personnel detection to obtain a personnel detection result of the target scene; adjusting the rotation angle of the camera based on the central point of the surrounding frame in the personnel detection result and the size information of the to-be-detected image; and determining whether to control a tracking device where the camera is located to move or not according to the area of the surrounding frame so as to track the personnel. According to the scheme disclosed by the invention, rapid personnel detection can be realized on embedded equipment without a neural network acceleration module, and a real-time personnel tracking effect is achieved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of image processing technology. More specifically, the present disclosure relates to a target detection and tracking method, an electronic device, and a computer-readable storage medium. Background Art

[0002] At present, personnel tracking is widely used in visual fields such as civil monitoring and urban monitoring. The existing methods for tracking people in videos mainly include the following three methods: Method 1: Use a network camera to transmit the collected video images to the GPU server in real time. The GPU server uses the personnel detection deep algorithm deployed on it to detect and track the people in the video. Method 2: Use an embedded device with a neural network acceleration function to transmit the collected video to the embedded processor. The embedded processor uses the neural network acceleration module to deploy a deep learning algorithm to detect and track people in the video. Method 3: Use traditional image processing algorithms on existing embedded devices to detect and track people.

[0003] However, the deployment costs and tracking effects of the three existing implementation methods are not ideal. Specifically, method one has high requirements for network bandwidth and network delay, resulting in poor real-time tracking effect. At the same time, the GPU server deployment cost is high. When there are few front-end network cameras, it is easy to waste GPU server resources. On the contrary, when there are many front-end network cameras, the concurrency is too large, which is easy to cause data congestion and poor real-time tracking effect. Method two requires that the embedded device be deployed with a neural network acceleration module, and the device deployment cost is relatively high. At the same time, for embedded devices that are not deployed with a neural network acceleration module, the original embedded device needs to be rectified to be compatible, which further increases the deployment cost. Furthermore, due to the limited image processing capabilities of embedded devices and the direct use of traditional image processing algorithms for personnel detection in dynamic video images, the accuracy of the detection results is low, resulting in poor real-time tracking effect of method three.

[0004] In view of this, there is an urgent need to provide a target detection and tracking solution so that fast personnel detection can be achieved on embedded devices that do not deploy neural network acceleration modules, thereby achieving real-time tracking effects. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes target detection and tracking solutions in the following aspects.

[0006] In a first aspect, the present disclosure provides a target detection and tracking method, comprising: using a camera to collect an image to be detected of a target scene containing a person; inputting the image to be detected into a target detection model for person detection to obtain a person detection result of the target scene, wherein the target detection model comprises a first detection head and a second detection head, the first detection head is used to detect 16-fold down-sampling features of the image to be detected, and the second detection head is used to detect 32-fold down-sampling features of the image to be detected; adjusting the rotation angle of the camera based on the center point of the bounding box in the person detection result and the size information of the image to be detected; and determining whether to control the movement of the tracking device where the camera is located according to the area of ​​the bounding box to achieve tracking of the person.

[0007] In some embodiments, based on the center point of the bounding box in the personnel detection result and the size information of the image to be detected, adjusting the rotation angle of the camera includes: determining the image center of the image to be detected based on the size information of the image to be detected; calculating the horizontal deviation and vertical deviation between the center point of the bounding box and the image center; and determining the horizontal movement angle and vertical movement angle of the camera based on the horizontal deviation and the vertical deviation.

[0008] In some embodiments, the horizontal movement angle and the vertical movement angle of the camera are determined based on the horizontal deviation and the vertical deviation in the following manner:

[0009]

[0010] Among them, θ X is the horizontal movement angle, R X is the camera's movable angle in the horizontal direction, D X is the horizontal deviation, W is the width of the image to be detected, θ Y is the vertical movement angle, R Y is the vertical movable angle of the camera, D Y is the vertical deviation, and H is the height of the image to be detected.

[0011] In some embodiments, after determining the horizontal movement angle and vertical movement angle of the camera based on the horizontal deviation and the vertical deviation, the method further includes: determining the horizontal movement direction and vertical movement direction of the camera based on the positional relationship between the center point of the bounding box and the center of the image.

[0012] In some embodiments, based on the positional relationship between the center point of the bounding box and the center of the image, determining the horizontal movement direction and the vertical movement direction of the camera includes: when the X coordinate of the center point is less than the X coordinate of the center of the image, determining that the horizontal movement direction of the camera is moving to the left; when the X coordinate of the center point is greater than the X coordinate of the center of the image, determining that the horizontal movement direction of the camera is moving to the right; when the Y coordinate of the center point is less than the Y coordinate of the center point of the image, determining that the vertical movement direction of the camera is moving upward; and when the Y coordinate of the center point is greater than the Y coordinate of the center point of the image, determining that the vertical movement direction of the camera is moving downward.

[0013] In some embodiments, determining whether to control the movement of a tracking device where the camera is located according to the area of ​​the bounding box includes: determining whether the area of ​​the bounding box is smaller than a preset area; and when the area of ​​the bounding box is smaller than the preset area, controlling the tracking device to move in the direction indicated by the camera.

[0014] In some embodiments, the output categories of the target detection model include people and non-people, the number of layers of the target detection model is 74, and the convolution kernel of the convolution layer of the target detection model is 3*3 or 1*1.

[0015] In some embodiments, the training process of the target detection model includes: obtaining positive sample images containing persons and negative sample images containing pseudo persons, wherein the pseudo persons are non-persons that are mistakenly identified as persons; respectively labeling the persons in the positive sample images and the pseudo persons in the negative sample images to form positive labeling information and negative labeling information; and inputting the positive sample images, positive labeling information, negative sample images and negative labeling information as training data into the target detection model to train it.

[0016] In a second aspect, the present disclosure provides an electronic device, comprising: a processor; and a memory storing program instructions for target detection and tracking, wherein when the program instructions are executed by the processor, the method described in the first aspect and its multiple embodiments are implemented.

[0017] In a third aspect, the present disclosure provides a computer-readable storage medium having program instructions for target detection and tracking stored thereon, which, when executed by a processor, implement the method described in the first aspect and its multiple embodiments.

[0018] Through the target detection and tracking scheme provided above, the target detection model of the disclosed embodiment can realize fast personnel detection on embedded devices without neural network acceleration modules. Subsequently, real-time personnel tracking can also be realized through the detected personnel detection results. Furthermore, the target detection and tracking scheme disclosed in the present invention can be run in ARM embedded devices with low processing performance, so that it can be applied to embedded devices such as fixed cameras, cloud billiard machines or inspection equipment, and has good portability. At the same time, there is no need to modify the embedded device, and it can run as long as the embedded device has an ARM processor, which can effectively reduce the cost of equipment deployment.

[0019] Furthermore, in some embodiments, the target detection model only detects two categories, personnel and non-personnel, which reduces the detection complexity, improves the detection efficiency, and helps to achieve real-time personnel tracking. At the same time, the target detection model has 74 layers, and the convolution kernel of the convolution layer is 3*3 or 1*1. Compared with the existing detection model, its number of layers and convolution kernel size are relatively low, which helps to reduce the number of parameters of the target detection model, thereby reducing the computational complexity and improving the detection efficiency.

[0020] Furthermore, in some embodiments, the target detection model includes a first detection head and a second detection head, the first detection head can downsample the image to be detected by 16 times, and the second detection head can downsample the image to be detected by 32 times. In other words, the target detection model disclosed in the present invention can extract features at two sizes of the image to be detected. Compared with the existing implementation method of extracting features at only one size, the features extracted by the present disclosure are richer, can prevent feature loss, and help improve the accuracy of personnel detection. Compared with the existing implementation method of extracting features at three sizes, the present disclosure can reduce the number of layers and parameters of the model, can ensure the detection speed, and improve the efficiency of personnel detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0022] Figure 1 is an exemplary flow chart of a target detection and tracking method according to some embodiments of the present disclosure;

[0023] Figure 2 is an exemplary flow chart of a method for processing an image to be detected by a target detection model according to an embodiment of the present disclosure;

[0024] Figure 3is an exemplary flow chart of a target detection and tracking method according to other embodiments of the present disclosure;

[0025] Figure 4 is a block diagram showing an exemplary structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0027] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0028] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0029] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0031] Exemplary Methods

[0032] Next, combine Figure 1The target detection and tracking method 100 of some embodiments of the present disclosure is exemplarily introduced. It is understandable that the method 100 can be executed by any appropriate embedded device with an ARM processor, such as but not limited to a fixed camera, a pan-tilt camera, and an inspection device.

[0033] like Figure 1 As shown, at step S101, method 100 can use the camera of the embedded device to collect the image of the target scene containing the person to be detected. The target scene disclosed in the present invention identifies the application scene of the embedded device, which can be a breeding scene, such as a pig house, a chicken house or a duck house, or a production scene, such as a food processing room or a supplies processing room.

[0034] It is understandable that the smaller the size of the image to be detected captured by the camera, the faster the detection speed, but the detection accuracy will be reduced. Conversely, the larger the size of the image to be detected captured by the camera, the higher the detection accuracy will be, but the detection speed will be reduced. Based on comprehensive consideration of the detection speed and detection accuracy, in the embodiment of the present disclosure, the size of the image to be detected can be 352*352.

[0035] Next, at step S102, the method 100 may input the image to be detected into the target detection model for personnel detection to obtain a personnel detection result of the target scene. The personnel detection result disclosed herein may include a bounding box of the person. The bounding box may be represented by the coordinates (X, Y), width, and height of the center point.

[0036] As known to those skilled in the art, the YoloV3 model is a relatively common and high-performance model in target detection models. However, the network model of the YoloV3 model has 252 layers, and applying it to an embedded device without a neural network acceleration module cannot achieve the effect of rapid detection and real-time tracking. Based on this, in the scheme disclosed herein, the YoloV3 model is improved to obtain a target detection model with 74 layers, and the convolution kernel of the convolution layer of the target detection model is 3*3 or 1*1. The reduction in the number of model layers leads to a reduction in the number of model parameters, and the detection speed of the model is improved, which makes it possible to apply the target detection model to an embedded device without a neural network acceleration module to perform personnel detection operations. In practical applications, the detection accuracy of the target detection model disclosed herein is two percentage points lower than that of the classic YoloV3 model, but the detection speed can be increased by about 10 times.

[0037] Furthermore, under the premise of ensuring the detection speed, in order to improve the detection accuracy, the target detection model disclosed in the present invention may include a first detection head and a second detection head. Here, the first detection head is used to detect the 16-fold downsampled features of the image to be detected, and the second detection head is used to detect the 32-fold downsampled features of the image to be detected, so that the features of the image to be detected at two sizes can be extracted to achieve multi-scale target detection. In addition, the detection categories of the target detection model disclosed in the present invention only include two categories: personnel and non-personnel. In other words, objects other than personnel are identified as non-personnel, which can reduce the detection complexity and improve the detection efficiency. In order to facilitate understanding of how the target detection model processes the image to be detected, it will be combined later. Figure 2 A method 200 for processing an image to be detected by a target detection model is introduced.

[0038] In the scheme disclosed in the present invention, when training the target detection model, positive sample images containing people and negative sample images containing pseudo-people can be first obtained. Here, pseudo-people are non-person objects that are mistakenly identified as people. Misidentification means that when performing personnel detection on the sample image, the object in the sample image is identified as a person. Then, the people in the positive sample image and the pseudo-people in the negative sample image can be labeled respectively by manual labeling, labeling tools, or a combination of the two to form positive labeling information and negative labeling information;. Finally, the positive sample image, positive labeling information, negative sample image, and negative labeling information can be input as training data into the target detection model to train it.

[0039] Next, at step S103, the method 100 can adjust the rotation angle of the camera based on the center point of the bounding box in the person detection result and the size information of the image to be detected. Specifically, the angle of the camera can be adjusted by adjusting the rotation angle of the pan / tilt where the camera is located.

[0040] In the scheme disclosed herein, based on the center point of the bounding box in the personnel detection result and the size information of the image to be detected, when adjusting the rotation angle of the camera, the image center of the image to be detected can be first determined based on the size information of the image to be detected. Then, the horizontal deviation and vertical deviation between the center point of the bounding box and the image center can be calculated. Finally, the horizontal movement angle and vertical movement angle of the camera can be determined based on the horizontal deviation and the vertical deviation.

[0041] In the solution disclosed herein, the following method is used to determine the horizontal movement angle and the vertical movement angle of the camera based on the horizontal deviation and the vertical deviation:

[0042]

[0043] Among them, θ Xis the horizontal movement angle, R X is the camera's movable angle in the horizontal direction, D X is the horizontal deviation, W is the width of the image to be detected, θ Y is the vertical movement angle, R Y is the vertical movable angle of the camera, D Y is the vertical deviation, and H is the height of the image to be detected.

[0044] It is understandable that those skilled in the art can select R according to actual needs. X and R Y The present disclosure does not make any specific restrictions on the specific value of. In one example, for a mobile gimbal, a horizontal movement of 90 degrees can cover the entire horizontal screen, and a vertical movement of 60 degrees can cover the entire vertical angle. In this case, R can be selected. X The value of is 90 degrees, R Y The value is 60 degrees.

[0045] In the scheme disclosed herein, after determining the horizontal movement angle and the vertical movement angle of the camera based on the horizontal deviation and the vertical deviation, method 100 can also determine the horizontal movement direction and the vertical movement direction of the camera based on the positional relationship between the center point of the bounding box and the center of the image. Specifically, when the X coordinate of the center point is less than the X coordinate of the center of the image, the horizontal movement direction of the camera can be determined to be moving to the left; when the X coordinate of the center point is greater than the X coordinate of the center of the image, the horizontal movement direction of the camera can be determined to be moving to the right; when the Y coordinate of the center point is less than the Y coordinate of the center point of the image, the vertical movement direction of the camera can be determined to be moving upward; and when the Y coordinate of the center point is greater than the Y coordinate of the center point of the image, the vertical movement direction of the camera can be determined to be moving downward.

[0046] Finally, at step S104, the method 100 may determine whether to control the tracking device where the camera is located to move according to the area of ​​the enclosing frame to achieve tracking of the person.

[0047] In the scheme disclosed in the present invention, the area of ​​the enclosing frame can be determined by the width and height of the enclosing frame. It can be understood that when the area of ​​the enclosing frame is small, it indicates that the distance between the person in the target scene and the camera is far. On the contrary, when the area of ​​the enclosing frame is large, it indicates that the distance between the person in the target scene and the camera is close. Based on this, in order to achieve the tracking of the person, when determining whether to control the movement of the tracking device where the camera is located according to the area of ​​the person's enclosing frame, it can be determined whether the area of ​​the person's enclosing frame is less than the preset area, and when the area of ​​the enclosing frame is less than the preset area, the tracking device can be controlled to move in the direction indicated by the camera. It can also be understood that those skilled in the art can select the specific value of the preset area according to actual needs, and the present disclosure does not make specific limitations on this.

[0048] Furthermore, during the movement of the tracking device, the camera can continuously collect images to be detected that contain people, and input the images to be detected into the target detection model to obtain people detection results. It can also continuously determine whether the area of ​​the enclosing box is smaller than the preset area. When the area of ​​the enclosing box is no longer smaller than the preset area, the tracking device can be controlled to stop moving.

[0049] Combination of the above Figure 1 A target detection and tracking method 100 is described. The method 100 can achieve fast personnel detection on an embedded device without a neural network acceleration module by utilizing a target detection model. Subsequently, real-time personnel tracking can also be achieved through the detected personnel detection results. In actual applications, on a 4-core 1.2G ARM processor, the method 100 can achieve a detection speed of 150ms per frame, and can achieve a real-time detection effect.

[0050] Furthermore, the target detection and tracking method disclosed in the present invention can be run in ARM embedded devices with low processing performance, so it can be applied to embedded devices such as fixed cameras, cloud billiard machines or inspection equipment, and has good portability. At the same time, there is no need to modify the embedded device, and it can run as long as the embedded device has an ARM processor, which can effectively reduce the cost of equipment deployment.

[0051] Figure 2 FIG. 2 is an exemplary flow chart showing a method 200 for processing an image to be detected by a target detection model according to an embodiment of the present disclosure. Figure 2 As shown, at step S201, the method 200 may input the image to be detected into the target detection model. Then, at step S202, the method 200 may perform a convolution operation on the image to be detected to obtain features of the image to be detected.

[0052] Next, at step S203 , the method 200 may downsample the features of the image to be detected by a factor of 16 to obtain the 16-fold downsampled features of the image to be detected.

[0053] Further, the 32-fold downsampled features of the image to be detected can be obtained. The method 200 can perform a convolution operation on the 16-fold downsampled features of the image to be detected at step S204 to obtain a new feature. Then, the new feature is downsampled 32 times at step S205 to obtain the 32-fold downsampled features of the image to be detected.

[0054] Next, at step S206, the method 200 may perform feature concatenation of the 16-fold down-sampling features and the 32-fold down-sampling features of the image to be detected to obtain multi-scale features of the image to be detected. Finally, at step S207, the method 200 may perform personnel detection based on the multi-scale features of the image to be detected to obtain a personnel detection result.

[0055] In actual operation, the image to be detected with the size of 352*352 is input into the target detection model, and the output obtained includes 22*22 and 11*11 grids, each of which can contain the confidence of 2 categories, the coordinates and confidence of 3 prediction boxes. The output dimensions are: 22*22*(2+3*5), 11*11*(2+3*5), where 2 is the number of categories, 3 is the number of prediction boxes, and 5 is the size of the prediction box (X, Y, width and height) and the parameter amount of confidence. Further, the two grids are converted and merged to obtain an output dimension of (22*22+11*11)*17=605*17.

[0056] Furthermore, the method 200 may first perform post-processing operations on the person detection result using non-maximum suppression and / or threshold screening to obtain a final person detection result.

[0057] Non-maximum suppression is to merge the bounding boxes whose intersection-union ratio in the person detection results is greater than a predetermined ratio. It should be noted that those skilled in the art can select the predetermined ratio according to actual needs, and this disclosure does not make specific comparisons on this.

[0058] The meaning of the intersection-over-union ratio (IOU) is the ratio of the intersection to the union of two regions, which can be expressed by the following formula:

[0059]

[0060] The larger the IOR, the higher the overlap between regions A and B. Therefore, when the IOR of two bounding boxes is higher than a certain threshold, it can be considered that the two bounding boxes correspond to the same person, so the two bounding boxes can be merged. Specifically, the merging method is to combine the two bounding boxes into one and retain the one with higher confidence. The bounding box after the IOR operation basically solves the problem of duplicate recognition and multiple labels.

[0061] Threshold screening is to use a predetermined threshold to perform secondary screening on the personnel detection results after non-maximum suppression, that is, the detection results of personnel with confidence levels lower than the predetermined threshold are considered to be unreliable and discarded, and the detection results of personnel with confidence levels higher than the predetermined threshold are retained as the final personnel detection results.

[0062] Figure 3 FIG. 1 is an exemplary flow chart showing a target detection and tracking method 100 according to some other embodiments of the present disclosure. Figure 3 As shown, before step S103, method 100 may also perform steps S105 and S106. Specifically, at step S105, method 100 may perform face recognition on the person in the image to be detected to obtain a face recognition result. The face recognition here may be implemented using an image matching algorithm or a lightweight face recognition model.

[0063] When face recognition is realized by using an image matching algorithm, the face image enclosed by the enclosing frame in the person detection result obtained in step S102 can be matched with the face image of the trusted person stored in advance to obtain the image matching degree as the face recognition result. In this case, for abnormal persons (i.e., untrustworthy persons), the obtained image matching degree will be very low.

[0064] When a lightweight face recognition model is used to implement face recognition, the face image enclosed by the enclosing frame in the person detection result obtained in step S102 can be input into the lightweight face recognition model to obtain the identity information of the person as the face recognition result. In this case, for abnormal persons, their identity information cannot be obtained, and preset identity information can be output, such as a stranger or a suspicious person.

[0065] Next, at step S106, method 100 can determine whether the person in the image to be detected is an abnormal person based on the face recognition result, and when it is determined that the person is an abnormal person, it can go to step S103 for execution. On the contrary, when it is determined that the person in the image to be detected is not an abnormal person, the person may not be tracked in real time. Specifically, when the image matching degree is less than the preset matching degree, the person can be determined to be an abnormal person, or when the identity information is the preset identity information, the person can be determined to be an abnormal person. It can be understood that those skilled in the art can determine the specific value of the preset matching degree according to actual needs, and the disclosed embodiment does not make specific limitations by comparison.

[0066] Next, combine Figure 4 An electronic device 400 provided by the embodiment of the present disclosure is exemplarily introduced. Figure 4 As shown, the electronic device 400 of the embodiment of the present disclosure may include a processor 401 , a memory 402 , and a communication bus 403 .

[0067] In the process of a specific embodiment, the processor 401 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the function of the processor may also be other, which is not specifically limited in this embodiment.

[0068] In the embodiment of the present disclosure, the communication bus 403 is used to realize the connection and communication between the processor 401 and the memory 402; the memory 402 stores program instructions for target detection and tracking; when the processor 401 executes the program instructions stored in the memory 402, the present disclosure is realized in combination with the attached Figures 1 to 3 The described target detection and tracking method.

[0069] Combination of the above Figure 4 An electronic device for target detection and tracking that can be used to perform the present disclosure is described. It should be understood that the device structure or architecture here is merely exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application. It is understandable that the description of each embodiment in the present disclosure emphasizes the differences between the various embodiments, and the same or corresponding parts can be referenced to each other. For the purpose of brevity, the present disclosure will not repeat them one by one.

[0070] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by a software program. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program instructions for target detection and tracking, and the program instructions can be used to implement the present disclosure in combination with the accompanying drawings. Figures 1 to 3 The described target detection and tracking method.

[0071] It should be noted that although the operations of the present method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0072] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

[0073] The collection and acquisition of various data in this disclosure complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is prohibited to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A target detection and tracking method, comprising: Using a camera to collect images of a target scene containing people to be detected; Input the image to be detected into a target detection model for personnel detection to obtain a personnel detection result of the target scene, wherein the target detection model includes a first detection head and a second detection head, the first detection head is used to detect the 16-fold down-sampled features of the image to be detected, and the second detection head is used to detect the 32-fold down-sampled features of the image to be detected; Adjusting the rotation angle of the camera based on the center point of the bounding box in the person detection result and the size information of the image to be detected; as well as Determine whether to control the tracking device where the camera is located to move according to the area of ​​the enclosing frame to achieve tracking of the person.

2. The method according to claim 1, wherein: Adjusting the rotation angle of the camera based on the center point of the bounding box in the person detection result and the size information of the image to be detected includes: Determine the image center of the image to be detected based on the size information of the image to be detected; Calculating the horizontal deviation and the vertical deviation between the center point of the bounding box and the center of the image; and The horizontal movement angle and the vertical movement angle of the camera are determined based on the horizontal deviation and the vertical deviation.

3. The method according to claim 2, wherein: The horizontal movement angle and the vertical movement angle of the camera are determined based on the horizontal deviation and the vertical deviation in the following manner: Among them, θ x is the horizontal movement angle, R X is the camera's movable angle in the horizontal direction, D X is the horizontal deviation, W is the width of the image to be detected, θ Y is the vertical movement angle, R Y is the vertical movable angle of the camera, D Y is the vertical deviation, and H is the height of the image to be detected.

4. The method according to claim 3, after determining the horizontal movement angle and the vertical movement angle of the camera based on the horizontal deviation and the vertical deviation, the method further comprises: Based on the positional relationship between the center point of the bounding box and the center of the image, the horizontal movement direction and the vertical movement direction of the camera are determined.

5. The method according to claim 4, wherein determining the horizontal movement direction and the vertical movement direction of the camera based on the positional relationship between the center point of the bounding box and the center of the image comprises: When the X coordinate of the center point is less than the X coordinate of the center of the image, determining that the horizontal movement direction of the camera is moving to the left; When the X coordinate of the center point is greater than the X coordinate of the center of the image, determining that the horizontal movement direction of the camera is moving rightward; When the Y coordinate of the center point is less than the Y coordinate of the center point of the image, determining that the vertical movement direction of the camera is upward movement; and When the Y coordinate of the center point is greater than the Y coordinate of the center point of the image, it is determined that the vertical movement direction of the camera is downward movement.

6. The method according to claim 1, wherein: Determining whether to control the tracking device where the camera is located to move according to the area of ​​the bounding box includes: Determining whether the area of ​​the enclosing frame is smaller than a preset area; and When the area of ​​the enclosing frame is smaller than the preset area, the tracking device is controlled to move along the direction indicated by the camera.

7. The method according to any one of claims 1 to 6, wherein: The output categories of the target detection model include personnel and non-personnel, the number of layers of the target detection model is 74, and the convolution kernel of the convolution layer of the target detection model is 3*3 or 1*1.

8. The method according to claim 7, wherein: The training process of the target detection model includes: Acquire a positive sample image containing a person and a negative sample image containing a pseudo-person, wherein the pseudo-person is a non-person who is mistakenly identified as a person; Respectively labeling the persons in the positive sample image and the pseudo persons in the negative sample image to form positive labeling information and negative labeling information; and The positive sample image, positive annotation information, negative sample image and negative annotation information are input into the target detection model as training data to train the target detection model.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for target detection and tracking, wherein when the program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-8.

10. A computer-readable storage medium storing program instructions for target detection and tracking, wherein when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images