Target detection method and device based on deep learning, equipment and storage medium
The deep learning-based method simplifies and adapts target detection models for financial surveillance by pruning layers and using lightweight functions, enhancing efficiency and accuracy in identifying targets in videos.
Patent Information
- Application Number
- CN202411254575.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-12-13
AI Technical Summary
Existing target detection models in financial transaction surveillance have complex structures and low accuracy due to their inability to perform targeted detection on multiple targets in videos, leading to inefficiencies.
A deep learning-based method that modifies an initial model according to task content by pruning unnecessary layers, using lightweight activation functions, and deploying on edge devices with asynchronous processing strategies to enhance model efficiency and accuracy.
The method simplifies model structure, reduces complexity, and improves target detection efficiency and accuracy by matching the model to specific tasks, enabling accurate identification of targets in videos.
Smart Images
Figure CN119131362A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing and financial technology, and in particular to a target detection method, device, computer equipment and storage medium based on deep learning. Background Art
[0002] With the development of computer vision technology, target detection has gradually been applied to the financial field to monitor the security of financial transaction places or transaction processes. For example, surveillance is set up in public places of banks or financial institutions to detect and identify targets in surveillance videos to analyze customer traffic in real time, monitor suspicious behaviors, and ensure the safety of personnel or transactions. However, the existing target detection model has a highly complex model structure and requires a large number of calculation processes. In addition, in practical applications, there may be multiple targets in a video. The existing target detection model cannot perform targeted target detection on the video to be detected according to the target detection task, resulting in low target detection accuracy. Therefore, how to improve the efficiency and accuracy of video target detection has become an urgent problem to be solved. Summary of the invention
[0003] The present application provides a deep learning-based target detection method, apparatus, computer equipment, and storage medium to improve the efficiency and accuracy of video target detection.
[0004] In a first aspect, the present application provides a target detection method based on deep learning, the method comprising:
[0005] When receiving a target detection task, obtaining a video to be detected and the task content of the target detection task;
[0006] Based on the task content, the preset initial model is modified to obtain the target detection model corresponding to the target detection task;
[0007] Based on the target detection model, target detection is performed on the video to be detected to obtain a target detection result of the target detection task.
[0008] In a second aspect, the present application also provides a target detection device based on deep learning, the device comprising:
[0009] A task content acquisition module is used to acquire the video to be detected and the task content of the target detection task when receiving the target detection task;
[0010] A target detection model acquisition module is used to modify the preset initial model based on the task content to obtain the target detection model corresponding to the target detection task;
[0011] The target detection result obtaining module is used to perform target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task.
[0012] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the deep learning-based target detection method as described above when executing the computer program.
[0013] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor enables the processor to implement the deep learning-based target detection method as described above.
[0014] The present application discloses a method, device, computer equipment and storage medium for target detection based on deep learning. When receiving a target detection task, the video to be detected and the task content of the target detection task are obtained; based on the task content, the preset initial model is modified to obtain the target detection model corresponding to the target detection task; based on the target detection model, the target detection is performed on the video to be detected to obtain the target detection result of the target detection task. The present application can modify the initial model according to the task content of the target detection task to simplify the model structure, reduce the complexity of the model for target detection, and match the simplified target detection model with the target detection task, so that the model can accurately identify the target in the video to be detected according to the target detection task, thereby improving the efficiency of target detection on the video to be detected and the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0016] Figure 1 is a schematic flow chart of a first embodiment of a deep learning-based target detection method provided in an embodiment of the present application;
[0017] Figure 2 is a schematic flow chart of a second embodiment of a deep learning-based target detection method provided in an embodiment of the present application;
[0018] Figure 3 is a schematic flow chart of a third embodiment of a target detection method based on deep learning provided by an embodiment of the present application;
[0019] Figure 4 A schematic block diagram of a deep learning-based target detection device provided in an embodiment of the present application;
[0020] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0022] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0023] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0024] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0025] The embodiments of the present application provide a method, device, computer equipment and storage medium for target detection based on deep learning. Among them, the target detection method based on deep learning can be applied to a server, and the initial model is modified according to the task content of the target detection task to simplify the model structure, reduce the complexity of the model for target detection, and match the simplified target detection model with the target detection task, so that the model can accurately identify the target in the video to be detected according to the target detection task, thereby improving the efficiency of target detection in the video to be detected and the accuracy of the detection results. Among them, the server can be an independent server or a server cluster.
[0026] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0027] See also Figure 1 , Figure 1 This is a schematic flow chart of a target detection method based on deep learning provided by an embodiment of the present application. The target detection method based on deep learning can be applied to a server to modify the initial model according to the task content of the target detection task to simplify the model structure, reduce the complexity of the model for target detection, and match the simplified target detection model with the target detection task, so that the model can accurately identify the target in the video to be detected according to the target detection task, thereby improving the efficiency of target detection in the video to be detected and the accuracy of the detection results. .
[0028] like Figure 1 As shown, the target detection method based on deep learning specifically includes steps S101 to S103.
[0029] S101, when receiving a target detection task, obtaining a video to be detected and the task content of the target detection task;
[0030] In one embodiment, when a target detection task is received, the target detection task is analyzed to obtain the task content, which may include the type of detection target (such as customers, pedestrians, vehicles or specific objects, etc.), the detection environment (such as indoors (such as banks, financial institutions trading halls), outdoors, etc.) or other task-related requirements.
[0031] In one embodiment, the video to be detected includes a real-time video stream, a surveillance video, etc.
[0032] In one embodiment, the target detection task may include a video to be detected, and may also specify how to obtain the video to be detected (such as an instruction to obtain the video to be detected) in the target detection task. Based on the target detection task, the video to be detected is obtained from a specified location.
[0033] S102, based on the task content, modifying the preset initial model to obtain a target detection model corresponding to the target detection task;
[0034] In one embodiment, the initial model can be a general target detection model, such as a convolutional neural network based on deep learning, a YOLOv8 model, etc.
[0035] In one embodiment, according to the task content of the target detection task, the initial model is modified to obtain a target detection model that meets the target detection task. Specifically, according to the task content, unnecessary layers and modules in the initial model are determined and deleted, the network depth of the backbone network CSPNet (Cross Stage Partial Network) is reduced, and lightweight activation functions and convolution functions are added to the initial model or replaced with the original activation functions and convolution functions in the initial model to reduce the number of parameters and complexity of the model calculation. The model is pruned and quantized, and trained using half-precision floating-point numbers to reduce the overall size of the model, reduce memory bandwidth and storage space, and ensure reasoning speed and prediction accuracy.
[0036] S103: Performing target detection on the video to be detected based on the target detection model to obtain a target detection result of the target detection task.
[0037] In one embodiment, the target detection model is used to perform target detection on each frame of the video to be detected, identify the target in the image and generate a detection bounding box, and record the target information detected in each frame and the corresponding detection bounding box coordinates.
[0038] In one embodiment, for the first frame in the video to be detected, the target tracking is initialized using the target detection result. For each subsequent frame, the target detected in the current frame is matched with the target tracked in the previous frame. This can be achieved by calculating the intersection-over-union ratio between bounding boxes, feature matching, or other tracking algorithms.
[0039] In one embodiment, the target's trajectory information is updated based on the matching results. If a new target is detected or an existing target disappears, the tracking list is updated. The trajectory information of each target in consecutive frames is recorded, including the target's identity, position change, and motion state.
[0040] In one embodiment, all matched and updated trajectory information is integrated to form a complete target tracking result. The target detection result is generated by combining the detection information and trajectory information of the target. Specifically, the trajectories of multiple detected targets are clustered or merged to eliminate duplicate detected targets, or to merge multiple detected targets into the same target.
[0041] Furthermore, before performing target detection on the video to be detected based on the target detection model and obtaining the target detection result of the target detection task, it also includes: based on a preset model deployment code, deploying the target detection model on preset edge devices respectively, and obtaining the device performance of the edge device; based on the device performance, determining the asynchronous processing strategy of the target detection task, so as to perform target detection on the video to be detected based on the asynchronous processing strategy.
[0042] In one embodiment, the obtained target detection model is exported to ONNX format, and the model is deployed on the edge device using a preset model deployment code (such as C++ code). It can be integrated with the device hardware accelerator to further improve the operating efficiency of the model.
[0043] In one embodiment, the performance of the edge device is obtained, specifically, the performance of the edge device is tested, including the performance of the CPU, GPU, memory, and storage. The power consumption when the model is running is measured, which is particularly important for battery-powered devices.
[0044] In one embodiment, the processing capability of the edge device is analyzed based on the acquired performance data, and an asynchronous processing strategy is formulated based on the processing capability of the edge device, such as adjusting the frame rate of the processed video to adapt to the processing speed of the device. Multiple frames of images are processed as a batch of inputs to improve efficiency. Priorities are set for different target detection tasks to ensure that key tasks are executed first.
[0045] In one embodiment, the target detection task may be multiple tasks, and the multiple tasks are asynchronously processed according to the asynchronous processing strategy. Specifically, a video stream is obtained from a video source and decomposed into separate frames. According to the asynchronous processing strategy, asynchronous target detection is performed on the video frames.
[0046] In one embodiment, the performance status of the edge device, such as CPU, GPU and memory usage, is monitored when the model is running. Based on the monitoring results, the asynchronous processing strategy is dynamically adjusted to optimize performance and resource usage.
[0047] The above-mentioned embodiments provide a method, apparatus, computer equipment and storage medium for target detection based on deep learning. When receiving a target detection task, the video to be detected and the task content of the target detection task are obtained; based on the task content, the preset initial model is modified to obtain the target detection model corresponding to the target detection task; based on the target detection model, the target detection is performed on the video to be detected to obtain the target detection result of the target detection task. The present application can modify the initial model according to the task content of the target detection task to simplify the model structure, reduce the complexity of the model for target detection, and match the simplified target detection model with the target detection task, so that the model can accurately identify the target in the video to be detected according to the target detection task, thereby improving the efficiency of target detection in the video to be detected and the accuracy of the detection results.
[0048] See also Figure 2 , Figure 2This is a schematic flow chart of a target detection method based on deep learning provided by an embodiment of the present application. The target detection method based on deep learning can be applied to a server to modify the initial model according to the task content of the target detection task to simplify the model structure, reduce the complexity of the model for target detection, and match the simplified target detection model with the target detection task, so that the model can accurately identify the target in the video to be detected according to the target detection task, thereby improving the efficiency of target detection in the video to be detected and the accuracy of the detection results.
[0049] like Figure 2 As shown, the target detection method based on deep learning specifically includes steps S201 to S203.
[0050] S201, based on the task content, determining a target module corresponding to the task content in each module of the initial model;
[0051] In one embodiment, based on the task content, it is determined which modules in the initial model are related to the task content, and the modules related to the task content are used as target modules. For example, if the task is to detect pedestrians, then the modules such as network layers and filters related to pedestrian detection are determined as target modules.
[0052] In a specific embodiment, the module parameters or inputs may be changed to obtain changes in the task results output by the initial model, determine the impact of the module on the results, and then evaluate the relevance of the module to the task content.
[0053] In another embodiment, when the initial model lacks modules related to the current task content, relevant modules are added to the initial model.
[0054] S202, based on the preset activation function, convolution function and the target module, modify the initial model to obtain a model to be trained;
[0055] In one embodiment, appropriate activation functions and convolution functions are selected according to the task content and task requirements, and the original activation functions and convolution functions in the initial model are replaced with the activation functions and convolution functions to reduce the number of parameters and complexity of the model calculations.
[0056] In one embodiment, the target module in the initial model is retained, the remaining modules are deleted, and the activation function and the convolution function are combined to obtain the model to be processed. The model to be processed is pruned and quantized to obtain the model to be trained.
[0057] Furthermore, based on the preset activation function, convolution function and the target module, the initial model is modified to obtain the model to be trained, including: based on the target module, determining the idle modules in each module of the initial model; deleting the idle modules from the initial model, and combining the activation function and the convolution function with the initial model to obtain the model to be processed; pruning and quantizing the model to be processed to obtain the model to be trained.
[0058] In one embodiment, the modules other than the target module in the initial model are regarded as idle modules, and the idle modules are deleted from the initial model.
[0059] In one embodiment, a suitable activation function is selected according to the task content. For example, the ReLU (Rect ifi ed Linear Unit) activation function is usually used to increase the nonlinearity of the model, while the Leaky ReLU (Leaky Rect ifi ed Linear Unit) can be used to solve the dead ReLU problem of ReLU. Specifically, the mapping relationship between the preset task and function can be found according to the task content to determine the activation function, or the activation function can be specified by the user.
[0060] In one embodiment, a suitable convolution function type is determined according to the task content, such as conventional convolution, depthwise separable convolution, or other special types of convolution.
[0061] In one embodiment, the selected activation function and convolution function are applied to relevant parts of the model to obtain a model to be processed to enhance the performance and efficiency of the model.
[0062] In one embodiment, model pruning is achieved by removing unimportant weights or neurons in the model to be processed to simplify the model structure. Specifically, the model can be pruned based on the importance of weights, a specific threshold of the Hessian matrix, or other pruning strategies.
[0063] In one embodiment, the weights and activations of the model to be processed are converted from floating point numbers to integer representations, such as 1 NT8 or 1 NT16, to achieve quantization of the model. This can significantly reduce the model size, increase the inference speed, and adapt to resource-constrained devices.
[0064] S203: training the model to be trained based on a preset training data set to obtain the target detection model.
[0065] In one embodiment, the modified model to be trained is trained using a preset training data set. The input data of the model (such as video frames) is passed through the model to be trained to generate output (such as target detection results). The error between the output and the true label is calculated, and the parameters of the model are updated using optimization algorithms such as gradient descent to reduce the error. After multiple iterations, when the performance of the model on the validation set is no longer significantly improved, the training process ends, and the model obtained at this time is the target detection model optimized for the current task.
[0066] See also Figure 3 , Figure 3 This is a schematic flow chart of a deep learning-based target detection method provided by an embodiment of the present application. The deep learning-based target detection method can be applied to a server to track a target in a video to be detected, obtain target trajectory information, and improve the efficiency and accuracy of target recognition results.
[0067] like Figure 3 As shown, the target detection method based on deep learning specifically includes steps S301 to S303.
[0068] S301, based on the target detection model, perform frame-by-frame detection on the video to be detected to obtain detection information of the target in the video to be detected and a detection bounding box of the target in each frame image;
[0069] In one embodiment, each frame of image is extracted from the video to be detected. Specifically, the video stream is decoded and converted into a separate image frame.
[0070] In one embodiment, each frame of the image is preprocessed to adapt to the input requirements of the target detection model, including adjusting the image size, normalizing pixel values, data enhancement, etc.
[0071] In one embodiment, the target detection model is used to perform target detection on each frame of the image, identify the target in the image and generate a detection bounding box, and record the target information detected in each frame and the corresponding bounding box coordinates.
[0072] S302, tracking the target based on the detection bounding box in each frame of the image to obtain trajectory information of the target;
[0073] In one embodiment, for the first frame in the video to be detected, the target tracking is initialized using the target detection result. For each subsequent frame, the target detected in the current frame is matched with the target tracked in the previous frame. This can be achieved by calculating the intersection-over-union ratio between bounding boxes, feature matching, or other tracking algorithms.
[0074] In one embodiment, the target's trajectory information is updated based on the matching results. If a new target is detected or an existing target disappears, the tracking list is updated. The trajectory information of each target in consecutive frames is recorded, including the target's identity, position change, and motion state.
[0075] Furthermore, the target is tracked based on the detection bounding box in each frame image to obtain trajectory information of the target, including: based on a Kalman filter algorithm and the detection bounding box of the target in the previous frame image, the trajectory of the target is predicted to obtain the predicted bounding box of the target in the current frame image; when the detection bounding box of the target in the current frame image is detected, a cost matrix is constructed based on the detection information of the target and the detection bounding box of the target; based on the Hungarian algorithm and the cost matrix, the target is tracked to obtain the trajectory information of the target.
[0076] In one embodiment, after the target is detected in the first frame image, the state of the Kalman filter is initialized using the detection bounding box of the target, including position, speed, etc.
[0077] In one embodiment, after the target is detected again in the second frame, the prediction step of the Kalman filter is used to estimate the position and state of the target in the next frame (i.e., the third frame). The Kalman filter is a recursive algorithm that can predict the position of the target in the third frame according to the motion information (such as speed, acceleration, etc.) of the target between consecutive frames to obtain a predicted bounding box.
[0078] In one embodiment, the predicted bounding box of the target in each subsequent frame of image can be predicted based on the detected bounding box of the previous frame of image and the trajectory information of the target recorded by the Kalman filter algorithm.
[0079] In one embodiment, a predicted bounding box of the target in the third frame image is obtained according to the prediction result of the Kalman filter.
[0080] In one embodiment, for each detected bounding box in the third frame, a matching cost between the detected bounding box and the predicted bounding box is calculated. The cost may be based on the intersection-over-union ratio, distance metric, or other correlation indicators of the bounding boxes.
[0081] In one embodiment, a cost matrix is constructed based on the matching costs of the detection bounding box and the prediction bounding box, wherein the cost matrix is usually a two-dimensional matrix, the rows represent the detection bounding boxes in the third frame, the columns represent the predicted trajectory states, and the elements in the matrix are the matching costs. The smaller the matching cost, the more matched the two are.
[0082] In one embodiment, the cost matrix is processed using the Hungarian algorithm to find the match that minimizes the total cost, that is, to determine the most likely predicted trajectory for each detection bounding box. According to the result of the Hungarian algorithm, the trajectory information of the target is updated, including the position and identity of the target in the third frame. The Hungarian algorithm is an algorithm for solving the best matching problem, which can find the best matching relationship between the predicted bounding box and the detected bounding box.
[0083] In one embodiment, when no detection bounding box matching the predicted bounding box is found in the Hungarian algorithm, the target is marked as lost. When a new detection bounding box is detected (ie, no predicted bounding box matches it), a new track record is started.
[0084] In one embodiment, all matched and updated trajectory information is integrated to form a complete target tracking result. Specifically, a target detection result is generated based on the detection information and trajectory information of the target. Specifically, the trajectories of multiple detected targets are clustered or merged to eliminate duplicate detected targets, or multiple detected targets are merged into the same target. Finally, the target detection result is output for subsequent analysis and application.
[0085] Furthermore, the target is tracked based on the Hungarian algorithm and the cost matrix to obtain the trajectory information of the target, including: obtaining a matching result between the detection bounding box and the prediction bounding box based on the Hungarian algorithm and the cost matrix; when the matching result is that the detection bounding box matches the prediction bounding box, updating the trajectory information of the target based on the Kalman filter algorithm until all frame images of the video to be detected are detected, and the trajectory information of the target is obtained.
[0086] In one embodiment, the cost matrix is processed using the Hungarian algorithm to find the match that minimizes the total cost, that is, to determine the most likely predicted trajectory for each detection bounding box. According to the result of the Hungarian algorithm, the trajectory information of the target is updated, including the position and identity of the target in the current frame. The Hungarian algorithm is an algorithm for solving the best matching problem, which can find the best matching relationship between the predicted bounding box and the detected bounding box.
[0087] In a specific embodiment, in each frame, the detection result of the current frame is matched with the prediction result, and the best match is found using the Hungarian algorithm. When the matching result is that the detection bounding box matches the prediction bounding box, the trajectory information of the target is updated based on the Kalman filter algorithm. Specifically, the Kalman filter will update the motion trajectory of the target based on the motion information (such as speed, acceleration, etc.) of the target between consecutive frames and the detection bounding box information in the current frame.
[0088] In one embodiment, when no detection bounding box matching the predicted bounding box is found in the Hungarian algorithm, the target is marked as lost. When a new detection bounding box is detected (ie, no predicted bounding box matches it), a new track record is started.
[0089] In one embodiment, the above matching and updating steps are repeated for each frame in the video to be detected until all frames are processed. During the processing, the trajectory information of each target is collected and stored. After the video processing is completed, the trajectory information of all targets is integrated to form a complete tracking result for each target.
[0090] In the above embodiment, the target is tracked according to the Hungarian algorithm and the Kalman filter algorithm to obtain the trajectory information of each target, with small calculation amount and low memory occupancy. At the same time, it has the ability to stably track multiple targets in real time, thereby improving the target detection efficiency and accuracy.
[0091] S303: Generate the target detection result based on the detection information of the target and the trajectory information.
[0092] In one embodiment, the detection information and trajectory information of all targets are integrated together to form a complete target detection result.
[0093] In a specific embodiment, the trajectories of the detected multiple targets are clustered or merged to eliminate repeated detected targets, or to merge the multiple detected targets into the same target. Finally, the target detection result is output for subsequent analysis and application.
[0094] See also Figure 4 , Figure 4 The embodiment of the present application provides a schematic block diagram of a deep learning-based target detection device, which is used to execute the aforementioned deep learning-based target detection method. The deep learning-based target detection device can be configured on a server.
[0095] like Figure 4 As shown, the deep learning-based target detection device 400 includes:
[0096] The task content acquisition module 401 is used to acquire the video to be detected and the task content of the target detection task when receiving the target detection task;
[0097] The target detection model acquisition module 402 is used to modify the preset initial model based on the task content to obtain the target detection model corresponding to the target detection task;
[0098] The target detection result obtaining module 403 is used to perform target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task.
[0099] Furthermore, the target detection model acquisition module 402 includes:
[0100] A target module determination unit, configured to determine, based on the task content, a target module corresponding to the task content in each module of the initial model;
[0101] A model to be trained obtaining unit, used to modify the initial model based on a preset activation function, a convolution function and the target module to obtain a model to be trained;
[0102] The target detection model acquisition unit is used to train the model to be trained based on a preset training data set to obtain the target detection model.
[0103] Furthermore, the to-be-trained model obtaining unit comprises:
[0104] An idle module determination subunit, used to determine an idle module in each module of the initial model based on the target module;
[0105] A model-to-be-processed obtaining subunit is used to delete the idle module from the initial model, and combine the activation function and the convolution function with the initial model to obtain the model to be processed;
[0106] The model to be trained obtaining subunit is used to perform pruning and quantization processing on the model to be processed to obtain the model to be trained.
[0107] Furthermore, the target detection result obtaining module 403 includes:
[0108] The target detection submodule is used to detect the video to be detected frame by frame based on the target detection model, and obtain the detection information of the target in the video to be detected and the detection boundary box of the target in each frame image;
[0109] A trajectory acquisition submodule, used to track the target based on the detection bounding box in each frame image to obtain the trajectory information of the target;
[0110] The detection result obtaining submodule is used to generate the target detection result based on the detection information of the target and the trajectory information.
[0111] Furthermore, the trajectory acquisition submodule includes:
[0112] A bounding box prediction unit, configured to predict the trajectory of the target based on a Kalman filter algorithm and a detection bounding box of the target in a previous frame image, and obtain a predicted bounding box of the target in a current frame image;
[0113] A cost matrix construction unit, configured to construct a cost matrix based on the detection information of the target and the detection bounding box of the target when the detection bounding box of the target in the current frame image is detected;
[0114] The trajectory information obtaining unit is used to track the target based on the Hungarian algorithm and the cost matrix to obtain the trajectory information of the target.
[0115] Furthermore, the trajectory information obtaining unit includes:
[0116] A matching result obtaining subunit, used for obtaining a matching result between the detection bounding box and the prediction bounding box based on the Hungarian algorithm and the cost matrix;
[0117] The trajectory information obtaining subunit is used to update the trajectory information of the target based on the Kalman filter algorithm when the matching result is that the detection boundary box matches the prediction boundary box, until all frame images of the video to be detected are detected, and obtain the trajectory information of the target.
[0118] Furthermore, the deep learning-based target detection device 400 further includes an asynchronous processing strategy generation module, and the asynchronous processing strategy generation module includes:
[0119] A device performance acquisition unit, configured to deploy the target detection model on preset edge devices based on a preset model deployment code, and acquire device performance of the edge devices;
[0120] The processing strategy determination unit is used to determine the asynchronous processing strategy of the target detection task based on the device performance, so as to perform target detection on the video to be detected based on the asynchronous processing strategy.
[0121] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0122] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.
[0123] See also Figure 5 , Figure 51 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.
[0124] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0125] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any target detection method based on deep learning.
[0126] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0127] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any target detection method based on deep learning.
[0128] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0129] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0130] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0131] When receiving a target detection task, obtaining a video to be detected and the task content of the target detection task;
[0132] Based on the task content, the preset initial model is modified to obtain the target detection model corresponding to the target detection task;
[0133] Based on the target detection model, target detection is performed on the video to be detected to obtain a target detection result of the target detection task.
[0134] In one embodiment, when the processor amends the preset initial model based on the task content to obtain the target detection model corresponding to the target detection task, it is used to implement:
[0135] Based on the task content, determining a target module corresponding to the task content in each module of the initial model;
[0136] Based on the preset activation function, convolution function and the target module, the initial model is modified to obtain a model to be trained;
[0137] The model to be trained is trained based on a preset training data set to obtain the target detection model.
[0138] In one embodiment, the processor, when correcting the initial model based on a preset activation function, a convolution function and the target module to obtain a model to be trained, is used to implement:
[0139] Based on the target module, determining an idle module in each module of the initial model;
[0140] Deleting the idle module from the initial model, and combining the activation function and the convolution function with the initial model to obtain a model to be processed;
[0141] Pruning and quantization are performed on the model to be processed to obtain the model to be trained.
[0142] In one embodiment, when the processor performs target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task, it is used to implement:
[0143] Based on the target detection model, the video to be detected is detected frame by frame to obtain detection information of the target in the video to be detected and a detection bounding box of the target in each frame image;
[0144] Based on the detection bounding box in each frame of the image, the target is tracked to obtain trajectory information of the target;
[0145] The target detection result is generated based on the detection information of the target and the trajectory information.
[0146] In one embodiment, when the processor tracks the target based on the detection bounding box in each frame of the image to obtain the trajectory information of the target, it is used to implement:
[0147] Based on the Kalman filter algorithm and the detection bounding box of the target in the previous frame image, the trajectory of the target is predicted to obtain the predicted bounding box of the target in the current frame image;
[0148] When a detection bounding box of the target in the current frame image is detected, a cost matrix is constructed based on the detection information of the target and the detection bounding box of the target;
[0149] Based on the Hungarian algorithm and the cost matrix, the target is tracked to obtain trajectory information of the target.
[0150] In one embodiment, when the processor implements tracking the target based on the Hungarian algorithm and the cost matrix to obtain the trajectory information of the target, it is used to implement:
[0151] Based on the Hungarian algorithm and the cost matrix, obtaining a matching result between the detection bounding box and the prediction bounding box;
[0152] When the matching result is that the detection bounding box matches the prediction bounding box, the trajectory information of the target is updated based on the Kalman filter algorithm until all frame images of the video to be detected are detected, and the trajectory information of the target is obtained.
[0153] In one embodiment, before the processor performs target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task, it is further used to implement:
[0154] Based on a preset model deployment code, the target detection model is deployed on a preset edge device respectively, and the device performance of the edge device is obtained;
[0155] Based on the device performance, an asynchronous processing strategy for the target detection task is determined, so as to perform target detection on the video to be detected based on the asynchronous processing strategy.
[0156] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the deep learning-based target detection methods provided in the embodiments of the present application.
[0157] The computer-readable storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.
[0158] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A target detection method based on deep learning, characterized in that: include: When receiving a target detection task, obtaining a video to be detected and the task content of the target detection task; Based on the task content, the preset initial model is modified to obtain the target detection model corresponding to the target detection task; Based on the target detection model, target detection is performed on the video to be detected to obtain a target detection result of the target detection task.
2. The target detection method based on deep learning according to claim 1, characterized in that: The step of modifying a preset initial model based on the task content to obtain a target detection model corresponding to the target detection task includes: Based on the task content, determining a target module corresponding to the task content in each module of the initial model; Based on the preset activation function, convolution function and the target module, the initial model is modified to obtain a model to be trained; The model to be trained is trained based on a preset training data set to obtain the target detection model.
3. The target detection method based on deep learning according to claim 2, characterized in that: The method of modifying the initial model based on the preset activation function, convolution function and target module to obtain a model to be trained includes: Based on the target module, determining an idle module in each module of the initial model; Deleting the idle module from the initial model, and combining the activation function and the convolution function with the initial model to obtain a model to be processed; Pruning and quantization are performed on the model to be processed to obtain the model to be trained.
4. The target detection method based on deep learning according to claim 1, characterized in that: The step of performing target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task includes: Based on the target detection model, the video to be detected is detected frame by frame to obtain detection information of the target in the video to be detected and a detection bounding box of the target in each frame image; Based on the detection bounding box in each frame of the image, the target is tracked to obtain trajectory information of the target; The target detection result is generated based on the detection information of the target and the trajectory information.
5. The target detection method based on deep learning according to claim 4, characterized in that: The step of tracking the target based on the detection bounding box in each frame of the image to obtain trajectory information of the target includes: Based on the Kalman filter algorithm and the detection bounding box of the target in the previous frame image, the trajectory of the target is predicted to obtain the predicted bounding box of the target in the current frame image; When a detection bounding box of the target in the current frame image is detected, a cost matrix is constructed based on the detection information of the target and the detection bounding box of the target; Based on the Hungarian algorithm and the cost matrix, the target is tracked to obtain trajectory information of the target.
6. The target detection method based on deep learning according to claim 5, characterized in that: The step of tracking the target based on the Hungarian algorithm and the cost matrix to obtain trajectory information of the target includes: Based on the Hungarian algorithm and the cost matrix, obtaining a matching result between the detection bounding box and the prediction bounding box; When the matching result is that the detection bounding box matches the prediction bounding box, the trajectory information of the target is updated based on the Kalman filter algorithm until all frame images of the video to be detected are detected, and the trajectory information of the target is obtained.
7. The target detection method based on deep learning according to any one of claims 1 to 6, characterized in that: Before performing target detection on the video to be detected based on the target detection model and obtaining the target detection result of the target detection task, the method further includes: Based on a preset model deployment code, the target detection model is deployed on a preset edge device respectively, and the device performance of the edge device is obtained; Based on the device performance, an asynchronous processing strategy for the target detection task is determined, so as to perform target detection on the video to be detected based on the asynchronous processing strategy.
8. A target detection device based on deep learning, characterized in that: include: A task content acquisition module is used to acquire the video to be detected and the task content of the target detection task when receiving the target detection task; A target detection model acquisition module is used to modify the preset initial model based on the task content to obtain the target detection model corresponding to the target detection task; The target detection result obtaining module is used to perform target detection on the video to be detected based on the target detection model to obtain the target detection result of the target detection task.
9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the deep learning-based target detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the deep learning-based target detection method according to any one of claims 1 to 7.