Target detection method and device, electronic equipment and computer readable medium

Abstract

The present disclosure relates to a target detection method and device, electronic equipment and computer readable medium. The method comprises: extracting a monitoring image from a monitoring video; performing data processing on the monitoring image to generate image data; inputting the image data into a target detection model to obtain at least one target position and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; and generating a target detection result based on the at least one target position and the at least one target category. The target detection method, device, electronic equipment and computer readable medium according to the present disclosure can accurately predict and identify multiple targets in an image while maintaining low complexity, so as to monitor the operation of power equipment.

Description

Technical Field

[0001] This disclosure relates to the field of computer information processing, and more specifically, to a target detection method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] With the rapid development of IoT technology, network devices in the IoT environment have accumulated massive amounts of data. Although cloud computing technologies have shown promising potential in processing this massive amount of data, it remains a challenge for many typical monitoring applications. These challenges stem primarily from two aspects. One significant challenge is how to process, dynamically transform, and analyze large amounts of data in real-time or near real-time. Data is growing at an unprecedented scale and speed, placing an increasingly heavy workload on communication networks for monitoring applications. Another challenging aspect is storage capacity. Typically, it's not necessary to store the vast amounts of raw data generated by monitoring applications entirely in the cloud for future analysis and processing. To reduce the storage costs of massive amounts of raw data, users often focus on storing analysis results based on the raw monitoring data, discarding this data once real-time analysis is complete.

[0003] Edge computing has received increasing attention in recent years in fields such as big data processing, cloud computing, and the Internet of Things (IoT), offering a feasible solution to the aforementioned challenges. Edge computing enables more monitoring application computational tasks to be executed on distributed nodes at the network edge, thus reducing latency and enabling real-time online decision-making. A typical example of a monitoring application is real-time status monitoring of power equipment in a power IoT environment. Since the power IoT deployed by power companies consists of a large number of power terminal devices, there is an urgent need for edge computing-based power monitoring application systems to analyze and process the monitoring image data of power equipment status collected by these terminal devices to monitor the condition of these devices. Summary of the Invention

[0004] In view of this, the present disclosure provides a target detection method, apparatus, electronic device and computer-readable medium that can accurately predict and identify multiple targets in an image while maintaining low complexity, so as to monitor the operation of power equipment.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a target detection method is proposed, the method comprising: extracting a surveillance image from a surveillance video; performing data processing on the surveillance image to generate image data; inputting the image data into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; and generating a target detection result based on the at least one target location and the at least one target category.

[0007] In one exemplary embodiment of this disclosure, the method further includes: labeling historical surveillance images to generate training set data; and training the target detection model using the training set data, a neural network with a Darknet-53 structure, and a logistic regression algorithm.

[0008] In one exemplary embodiment of this disclosure, the method further includes: setting a positive sample threshold; evaluating the computational performance of the target detection model using the positive sample threshold; and correcting the target detection model when the evaluation performance does not meet the policy requirements.

[0009] In one exemplary embodiment of this disclosure, labeling historical surveillance images to generate training image set data includes: identifying targets in the historical surveillance images; labeling the location of the targets; and / or labeling the category of the targets; and / or labeling the size of the targets.

[0010] In one exemplary embodiment of this disclosure, the target detection model is generated by training the training set data, a Darknet-53 neural network, and a logistic regression algorithm. The process includes: inputting the training set data into a Darknet-53 neural network for feature extraction; performing calculations on the training set data using an initial detection model based on multiple convolutional layers and multiple residual network layers; performing residual training on the truncated results using short connections; and generating the target detection model when the training results satisfy a loss function, wherein a normally distributed loss function is used for optimization.

[0011] In one exemplary embodiment of this disclosure, data processing of the monitoring image to generate image data includes: performing image restoration processing on the monitoring image; and / or performing image enhancement processing on the monitoring image.

[0012] In one exemplary embodiment of this disclosure, inputting the image data into a target detection model to obtain at least one target location and at least one target category includes: inputting the image data into a Darknet-53 structured neural network in the target detection model to generate image feature data; generating at least one target bounding box based on the feature data; determining the at least one target location based on the at least one target bounding box; and determining the at least one target category of the object in the at least one target bounding box using a logistic regression algorithm in the target detection model.

[0013] According to one aspect of this disclosure, a target detection device is proposed, comprising: an image module for extracting surveillance images from surveillance video; a processing module for processing the surveillance images to generate image data; a calculation module for inputting the image data into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; and a target module for generating target detection results based on the at least one target location and the at least one target category.

[0014] According to one aspect of this disclosure, an electronic device is proposed, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.

[0015] According to one aspect of this disclosure, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described above.

[0016] According to the target detection method, apparatus, electronic device, and computer-readable medium disclosed herein, a monitoring image is extracted from a surveillance video; the monitoring image is processed to generate image data; the image data is input into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; the method of generating target detection results based on the at least one target location and the at least one target category can accurately predict and identify multiple targets in an image while maintaining low complexity, so as to monitor the operation of power equipment.

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0018] The above and other objects, features, and advantages of this disclosure will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this disclosure, and those skilled in the art will be able to obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 This is a flowchart illustrating a target detection method according to an exemplary embodiment.

[0020] Figure 2 This is a flowchart illustrating a target detection method according to an exemplary embodiment.

[0021] Figure 3 This is a flowchart illustrating a target detection method according to another exemplary embodiment.

[0022] Figure 4 Comparison of average accuracy (AP).

[0023] Figure 5 Comparison of mean precision (mAP).

[0024] Figure 6 Average precision (AP) for datasets of different sizes within the automotive category.

[0025] Figure 7 Average precision (AP) of datasets of different sizes within the non-motorized vehicle category.

[0026] Figure 8 Average precision (AP) for different sizes of datasets under the pedestrian category.

[0027] Figure 9 The average precision (AP) for different classes in a dataset of 26,000.

[0028] Figure 10 The average precision (AP) for different classes in a dataset of 32,500.

[0029] Figure 11 The average precision (AP) for different classes in a dataset of 39,000.

[0030] Figure 12 This is a block diagram illustrating a target detection device according to an exemplary embodiment.

[0031] Figure 13 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0032] Figure 14 This is a block diagram illustrating a computer-readable medium according to an exemplary embodiment. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0034] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] It should be understood that while the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this disclosure. As used herein, the term "and / or" includes all combinations of any and more of the associated listed items.

[0038] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing this disclosure, and therefore cannot be used to limit the scope of protection of this disclosure.

[0039] This disclosure addresses the problems mentioned in the background and proposes an object detection method that can be used as an edge service to process and analyze massive image data collected by multiple terminal devices. A lightweight multi-layer convolutional neural network, Darknet-53, is used to extract multi-target features from the training dataset and predict multiple targets in the image to achieve the detection of target objects in surveillance images. This disclosure also proposes a training dataset preprocessing method for image filtering, image enhancement, and manual category labeling, as well as an optimization method using a normally distributed loss function.

[0040] Figure 2 This is a flowchart illustrating a target detection method according to an exemplary embodiment. The target detection method 20 includes at least steps S202 to S208.

[0041] like Figure 2 As shown, in S202, a monitoring image is extracted from the monitoring video. For example, a monitoring image can be extracted from the monitoring video of power equipment.

[0042] In S204, data processing is performed on the monitored image to generate image data. For example, image restoration processing may be performed on the monitored image; and / or image enhancement processing may be performed on the monitored image.

[0043] In step S206, the image data is input into a target detection model to obtain at least one target location and at least one target category. The target detection model is generated based on a Darknet-53 neural network and a logistic regression algorithm. For example, the image data is input into the Darknet-53 neural network in the target detection model to generate image feature data; at least one target bounding box is generated based on the feature data; the at least one target location is determined based on the at least one target bounding box; and the at least one target category of the object within the at least one target bounding box is determined using the logistic regression algorithm in the target detection model.

[0044] In S208, a target detection result is generated based on the at least one target location and the at least one target category.

[0045] According to the target detection method disclosed herein, a monitoring image is extracted from a surveillance video; the monitoring image is processed to generate image data; the image data is input into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; the method of generating target detection results based on the at least one target location and the at least one target category can accurately predict and identify multiple targets in an image while maintaining low complexity, so as to monitor the operation of power equipment.

[0046] It should be clearly understood that this disclosure describes how specific examples are formed and used, but the principles of this disclosure are not limited to any details of these examples. Rather, based on the teachings of this disclosure, these principles can be applied to many other embodiments.

[0047] Figure 2 This is a flowchart illustrating a target detection method according to another exemplary embodiment. Figure 2 The process shown in step 20 is a detailed description of “training and generating an object detection model”.

[0048] like Figure 2 As shown, in S202, historical surveillance images are labeled to generate training set data. This may include, for example, identifying targets in the historical surveillance images; labeling the locations of the targets; and / or labeling the categories of the targets; and / or labeling the sizes of the targets.

[0049] In one specific embodiment, the Label Image tool can be used to manually annotate the raw image data. After manual annotation, the name, specific location information, and size of the target category of interest in each image will be obtained. The files annotated by the LabelImage tool will be stored in XML format, and all sample data will be converted into a dataset in VOC format that can be accepted by the training network.

[0050] In S204, the target detection model is generated by training the training set data, a Darknet-53 neural network, and a logistic regression algorithm. For example, the training set data can be input into a Darknet-53 neural network for feature extraction; an initial detection model can be calculated based on the training set data using multiple convolutional layers and multiple residual network layers; residual training can be performed on the truncated results using short connections; and the target detection model is generated when the training result satisfies the loss function, wherein a normally distributed loss function is used for optimization.

[0051] In S206, a positive sample threshold is set. The threshold can be, for example, 0.5.

[0052] In S208, the computational performance of the target detection model is evaluated using the positive sample threshold. If the evaluation performance does not meet the strategy, the target detection model is corrected. Samples that meet the following conditions can be designated as positive samples:

[0053]

[0054] Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, B is the bounding box predicted by the detection model, and Bg is the manually labeled ground truth bounding box.

[0055] This disclosed object detection method, through a series of preprocessing techniques, artificially establishes a road traffic dataset for model training and testing. Extensive experiments verify the effectiveness and efficiency of this method for real-time, massive data processing and analysis. It can serve as an edge service for real-time multi-object detection, such as road image object detection and power state analysis based on power equipment image object detection, particularly achieving very high model prediction accuracy while reducing model training time.

[0056] Figure 3 This is a flowchart illustrating a target detection method according to another exemplary embodiment. Figure 3 The process shown in step 30 is a detailed description of "training and generating object detection models" from another perspective.

[0057] like Figure 3 As shown in step S302, image screening involves first acquiring traffic videos and selecting suitable research samples. Then, video framing technology is used to convert the acquired videos into JPG still images.

[0058] In S304, image enhancement is performed. Image enhancement techniques are used to preprocess the data.

[0059] In S306, the training dataset is constructed. Images are manually labeled to obtain the training set for road object detection.

[0060] In S308, feature extraction is performed based on the Darknet-53 network structure.

[0061] Feature extraction can be performed using 75 layers, including 53 convolutional layers and the rest Res layers. The input image for each convolutional layer is 416×416 pixels with 3 channels, each convolutional kernel is 3×3, and the stride is 1. The output of each convolutional layer is a 416×416 feature map with 32 channels. The input and output of the Res layers are generally consistent; they perform no operations other than calculating the loss value. The core idea of ​​the Res layers is to divide the deep neural network into several sub-networks, each containing fewer layers, and then use a shortcut connection method to perform residual training on each sub-network. Each sub-network learns a portion of the total loss, ultimately achieving a smaller overall loss, thus effectively controlling gradient propagation and avoiding gradient vanishing or exploding, which are detrimental to training. The core idea of ​​this model is to divide each training image into an A×A grid. Each grid predicts B object probabilities and C class probabilities. This includes 24 convolutional layers and 2 fully connected layers, using 1×1 and 3×3 convolutional kernels as initial modules.

[0062] In S310, logistic regression is used to predict classification.

[0063] Considering the characteristics of the collected traffic target dataset, the output dimension of the model in this paper is 26×26×(3×5+3), with 3 categories representing cars, pedestrians, and non-motorized vehicles. Logistic regression is used to predict the object category. The calculation of the relevant coordinates of the predicted bounding box of the detected target is shown in the formula:

[0064] x=σ(t x )+c x ;

[0065] y=σ(t y )+c y ;

[0066]

[0067] h = ρ h e th ;

[0068] Where x, y, w, and h represent the center coordinates of the prediction box and the width and height c of the prediction box, respectively. x and c y Let t be the coordinate of the top-left corner of the grid. x ,t y ,t w ,t h ρ represents the predicted center coordinates, width, and height of the grid bounding box. w and ρ h Let be the width and height of the grid, and σ be the sigmoid function.

[0069] In S312, a loss function with a normal distribution is used for optimization.

[0070] The loss function consists of three parts: coordinate error, confidence error, and classification error. The model parameters are optimized by calculating the mean square sum of the errors of the network output (A×A×(B×5+C)) dimensional vector and the true target input (A×A×(B×5+C)) dimensional vector. This means that if the target appears in cell i, it is marked as 1, otherwise it is marked as 0. and These represent whether the target appears in or does not appear in the j-th boundary of the i-th cell.

[0071] (w,h,x,y) represents the coordinate error, where w and h represent the width and height of the predicted bounding box, x and y represent the center coordinates of the predicted bounding box, and λ represents the coordinate error. c This is the weighting constant.

[0072]

[0073] The confidence error is shown in the formula above, where Loss2 represents the confidence level when the grid contains the target, Loss3 represents the confidence level when the grid does not contain the target, and λ nobject F is the weighting constant. i This represents the confidence level.

[0074]

[0075]

[0076] The class error is shown in the formula above, ρ i (c) represents the classification probability:

[0077]

[0078] The overall loss function error is expressed as shown in the formula above:

[0079] Loss=Loss1+Loss2+Loss3+Loss4

[0080] To make the loss function converge as quickly as possible, weight parameter optimization methods can be used to adjust the weight parameter λ. c and λ nobject Choose an appropriate value, as follows:

[0081]

[0082] Where λ∈Λ represents a set of hyperparameters, here it is λ c and λ nobject t represents the number of iterations, and f(λ) represents the loss function Loss. It follows a normal distribution with respect to λ. Φ represents the cumulative distribution function.

[0083] F x (x)=P(X≤x)σ represents the probability density function

[0084] f X (x)≥0

[0085]

[0086] As shown above, through continuous iteration, the optimal values ​​of the parameters are finally obtained, so that the loss function reaches its minimum within a certain range, and the overall optimization of the model is achieved.

[0087] In S314, the efficiency of multi-target detection is evaluated.

[0088] The experiments were conducted using Ubuntu 16.04, a GPU with a GTX 1070 and 8GB of RAM, and CUDA / CUDNN. The training dataset was the artificially constructed dataset mentioned above, containing 40,765 images for cars, pedestrians, and non-motorized vehicles. The existing dataset was divided into six training datasets with different numbers of images: 6,500, 13,000, 19,500, 26,000, 32,500, and 39,000. The remaining 1,765 images were used for model testing.

[0089] The performance of multi-target detection can be evaluated using average precision (AP) and mean average precision (mAP), as shown in the following formula.

[0090]

[0091] AP C It is the average precision of category C.

[0092]

[0093] mAP is the ratio of the sum of all average precisions for each class to the total number of classes.

[0094] Among them, P C This is the precision for class C, representing the percentage of samples predicted as positive that are actually positive. The definition of class C precision is as follows.

[0095]

[0096] Among them, TP C It was originally a positive sample object, but the detection model further classified it as a positive sample; FP C The object was originally a negative sample, but the detection model incorrectly classified it as a positive sample.

[0097] This disclosure will also validate the object detection method from two aspects: effectiveness and efficiency. The effectiveness analysis, for a given road traffic object recognition task, uses AP and mAP as indicators to verify whether the method of this disclosure can achieve better classification performance based on object detection. Here, this paper will compare and analyze the introduced method with the currently popular YOLOv2 method. On the other hand, the efficiency analysis aims to verify whether the method can meet the requirements of real-time object detection.

[0098] 1) Validity Analysis

[0099] In this experiment, network models were trained on the training set using the network structures proposed in this paper and the YOLOv2 method, respectively, to detect three types of targets: vehicles, pedestrians, and non-motorized vehicles. Then, their AP and mAP were compared.

[0100] The experiment was conducted on a dataset containing 39,000 training instances for approximately 45,000 iterations. Figure 4 and Figure 5 Experimental results for AP values ​​of two different network structures, one based on YOLOv2 and the other based on the method described in this paper, are shown for each class (car, pedestrian and non-motorized vehicle).

[0101] like Figure 4 As shown, the method presented in this paper achieves higher average accuracy than the YOLOv2-based method in each category (cars, pedestrians, and motor vehicles). This demonstrates that training the model using the Darknet-53 network architecture is an ideal method for achieving high-precision multi-object detection.

[0102] Furthermore, the mAP value of the traffic multi-object detection method based on the method presented in this paper is significantly higher than that of the object detection method based on YOLOv2. In summary... Figure 4 and Figure 5 Based on comprehensive analysis, the method used in this paper is more effective than previous road multi-target detection methods.

[0103] 2) Efficiency Analysis

[0104] The purpose of efficiency analysis is to determine whether the model used in this paper can quickly converge to high accuracy during the model training process. This will be judged from two perspectives: the convergence speed of the average accuracy of each class, and the convergence speed of the average accuracy of training sets of different sizes.

[0105] Convergence speed of AP under different dataset sizes.

[0106] This disclosure first verifies whether the method presented in this paper can achieve very high average accuracy on different types of datasets, including those of cars, pedestrians, and non-motorized vehicles. In the experiments, the original dataset was divided into six subsets of different sizes: 6500, 13000, 19500, 26000, 32500, and 39000. The experimental results are as follows: Figures 6 to 8 As shown in these three figures, for datasets of different sizes, the AP values ​​for all three types increase rapidly with the number of iterations. When the number of iterations reaches approximately 10,000, the change in AP values ​​tends to level off, and the classification performance becomes relatively stable. This means that the model used can achieve a high average accuracy, and the model's loss function tends to converge. This clearly reflects that the method used is effective for real-time multi-object road object detection.

[0107] Convergence speed of AP under different categories.

[0108] Next, this disclosure verifies whether the method presented in this paper can achieve high average accuracy across three different categories: cars, pedestrians, and non-motorized vehicles. Experimental results are as follows: Figures 9 to 11 As shown in the figures, we typically choose datasets of sizes 26,000, 32,500, and 39,000 for our experiments. As these three figures illustrate, the AP values ​​on the three subsets increase significantly with the number of iterations. When the number of iterations reaches 5,000–10,000, the change in AP values ​​tends to level off, and the classification performance becomes relatively stable. This means that when the model's loss function converges, the model can achieve a high average accuracy, proving that this method can be used for effective model training.

[0109] In summary, through comprehensive analysis of the above results, it can be found that with the increase of the number of iterations, the method disclosed in this paper can converge quickly and achieve a high average accuracy. This method is very suitable for model training in edge computing environments that require real-time processing and analysis. Therefore, the method adopted in this paper is effective for real-time multi-object detection applications such as road image object detection and power status analysis based on power equipment image object detection, especially in achieving very high model prediction accuracy while reducing model training time.

[0110] This disclosure first collects traffic videos and selects suitable research samples from them. Then, it uses video framing technology to convert the collected videos into JPG still images. Next, image enhancement techniques are used to preprocess the data, and the images are manually labeled to obtain a training set for road target detection. Then, the proposed image multi-target classification method is used to train the traffic target model on the constructed dataset, performing feature extraction, prediction classification, and loss function optimization. Finally, relevant experiments are conducted to verify the proposed method and evaluate the efficiency of multi-target detection, thus demonstrating the feasibility of the method disclosed in this disclosure.

[0111] Those skilled in the art will understand that all or part of the steps of the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, it performs the functions defined by the methods provided in this disclosure. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0112] Furthermore, it should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0113] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0114] Figure 12 This is a block diagram illustrating a target detection device according to an exemplary embodiment. Figure 12 As shown, the target detection device 120 includes: an image module 1202, a processing module 1204, a calculation module 1206, and a target module 1208.

[0115] Image module 1202 is used to extract monitoring images from monitoring video;

[0116] Processing module 1204 is used to process the monitoring image to generate image data;

[0117] The calculation module 1206 is used to input the image data into the target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm;

[0118] The target module 1208 is used to generate target detection results based on the at least one target location and the at least one target category.

[0119] According to the target detection device of this disclosure, a monitoring image is extracted from a surveillance video; the monitoring image is processed to generate image data; the image data is input into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; the method of generating target detection results based on the at least one target location and the at least one target category can accurately predict and identify multiple targets in an image while maintaining low complexity, so as to monitor the operation of power equipment.

[0120] Figure 13 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0121] The following reference Figure 13 To describe an electronic device 1300 according to such an embodiment of the present disclosure. Figure 13 The electronic device 1300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0122] like Figure 13As shown, the electronic device 1300 is presented in the form of a general-purpose computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, a bus 1330 connecting different system components (including storage unit 1320 and processing unit 1310), a display unit 1340, etc.

[0123] The storage unit stores program code that can be executed by the processing unit 1310, causing the processing unit 1310 to perform the steps described in this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1310 can perform actions such as... Figure 1 , Figure 2 , Figure 3 The steps are shown in the figure.

[0124] The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 13201 and / or a cache storage unit 13202, and may further include a read-only memory unit (ROM) 13203.

[0125] The storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0126] Bus 1330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0127] Electronic device 1300 can also communicate with one or more external devices 1300' (e.g., keyboard, pointing device, Bluetooth device, etc.), enabling users to communicate with devices that interact with electronic device 1300, and / or any device that allows electronic device 1300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1350. Furthermore, electronic device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1360. Network adapter 1360 can communicate with other modules of electronic device 1300 via bus 1330. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 14 As shown, the technical solution according to the embodiments of this disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of this disclosure.

[0129] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0131] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0132] The aforementioned computer-readable medium carries one or more programs that, when executed by a device, cause the computer-readable medium to perform the following functions: extracting surveillance images from surveillance video; processing the surveillance images to generate image data; inputting the image data into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; and generating target detection results based on the at least one target location and the at least one target category.

[0133] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0134] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0135] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A target detection method, characterized in that, include: Extracting surveillance images from surveillance video; The monitored images are processed to generate image data; The image data is input into the target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a neural network with a Darknet-53 structure and a logistic regression algorithm; Generate target detection results based on the at least one target location and the at least one target category; Also includes: Historical surveillance images are labeled to generate training set data; The target detection model is generated by training the training set data, using a Darknet-53 neural network, and a logistic regression algorithm. The target detection model is generated by training the training set data, a Darknet-53 neural network, and a logistic regression algorithm, including: The training set data is input into a Darknet-53 neural network for feature extraction. The initial detection model is based on multiple convolutional layers and multiple residual network layers to compute on the training set data; The calculation results are used for residual training via short connections; When the training results satisfy the loss function, the target detection model is generated, wherein a normally distributed loss function is used for optimization. The loss function consists of three parts: coordinate error, confidence error, and classification error. Overall loss function error representation: ； Loss represents the overall loss function error, Loss1 represents the coordinate error, Loss2 represents the confidence level when the grid contains the target, Loss3 represents the confidence level when the grid does not contain the target, and Loss4 represents the classification error. ； The coordinate error is represented by w and h, where w and h represent the width and height of the predicted bounding box, and x and y represent the center coordinates of the predicted bounding box. This is the weighting constant; ； ； Loss2 represents the confidence level when the grid contains the target, and Loss3 represents the confidence level when the grid does not contain the target. Here is the weighting constant. Represents confidence level; ； This represents the classification probability.

2. The method as described in claim 1, characterized in that, Also includes: Set a positive sample threshold; The computational performance of the target detection model is evaluated using the positive sample threshold. If the evaluation results do not meet the strategy, the target detection model is modified.

3. The method as described in claim 1, characterized in that, Historical surveillance images are labeled to generate training image set data, including: Identify targets in historical surveillance images; Mark the location of the target; and / or Label the category of the target; and / or The size of the target is labeled.

4. The method as described in claim 1, characterized in that, The process of processing the monitored images to generate image data includes: Perform image restoration processing on the monitored images; and / or The monitoring image is then subjected to image enhancement processing.

5. The method as described in claim 1, characterized in that, The image data is input into the target detection model to obtain at least one target location and at least one target category, including: The image data is input into the Darknet-53 neural network in the target detection model to generate image feature data; At least one target bounding box is generated based on the feature data; The location of the at least one target is determined based on the at least one target bounding box; The at least one target category of the object in the at least one target bounding box is determined by the logistic regression algorithm in the target detection model.

6. A target detection device, characterized in that, include: The image module is used to extract surveillance images from surveillance video; The processing module is used to process the monitored images to generate image data; A calculation module is used to input the image data into a target detection model to obtain at least one target location and at least one target category, wherein the target detection model is generated based on a Darknet-53 neural network and a logistic regression algorithm; it also includes: Historical surveillance images are labeled to generate training set data; The target detection model is generated by training the training set data, using a Darknet-53 neural network, and a logistic regression algorithm. The target detection model is generated by training the training set data, a Darknet-53 neural network, and a logistic regression algorithm, including: The training set data is input into a Darknet-53 neural network for feature extraction. The initial detection model is based on multiple convolutional layers and multiple residual network layers to compute on the training set data; The calculation results are used for residual training via short connections; When the training results satisfy the loss function, the target detection model is generated, wherein a normally distributed loss function is used for optimization. The loss function consists of three parts: coordinate error, confidence error, and classification error. Overall loss function error representation: ； Loss represents the overall loss function error, Loss1 represents the coordinate error, Loss2 represents the confidence level when the grid contains the target, Loss3 represents the confidence level when the grid does not contain the target, and Loss4 represents the classification error. ； The coordinate error is represented by w and h, where w and h represent the width and height of the predicted bounding box, and x and y represent the center coordinates of the predicted bounding box. This is the weighting constant; ； ； Loss2 represents the confidence level when the grid contains the target, and Loss3 represents the confidence level when the grid does not contain the target. Here is the weighting constant. Represents confidence level; ； Represents the probability of classification; The target module is used to generate target detection results based on the at least one target location and the at least one target category.

7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images