Mobile target detection method and device, storage medium and electronic device

Abstract

The application discloses a mobile target detection method and device, a storage medium and an electronic device. The method comprises the following steps: acquiring a mobile video stream containing a mobile target, wherein the mobile video stream comprises a current frame image of a current moment and a historical frame image before the current moment; inputting the historical frame image into a background modeling module in a mobile target detection model; performing background modeling on the historical frame image based on the background modeling module and outputting a background image; inputting the background image and the current frame image into a detection module in the mobile target detection model; and outputting a mobile target detection result corresponding to the mobile target. The detection module is used for detecting the background image and the current frame image. The mobile target detection method provided by the application realizes the detection of the mobile target in the mobile video stream.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a moving target detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] Moving target detection in videos has a wide range of applications in intelligent monitoring, human-computer interaction, behavior analysis, intrusion detection, anomaly detection, object tracking, and autonomous driving. Moving target detection aims to locate and classify specified moving targets in videos. SUMMARY

[0003] Embodiments of the present application provide a moving target detection method, device, storage medium and electronic device, which can provide moving target detection for video streams. The technical solution is as follows:

[0004] In a first aspect, the embodiments of the present application provide a moving target detection method, which comprises:

[0005] Obtaining a moving video stream containing a moving target, the moving video stream comprising a current frame image at a current time and a historical frame image before the current time;

[0006] Inputting the historical frame image into a background modeling module in a moving target detection model to obtain a background image, the background modeling module being configured to perform average pooling operations on the historical frame image using at least two pooling kernels with different receptive fields to obtain at least two feature maps, and performing fusion processing on the at least two feature maps;

[0007] Inputting the background image and the current frame image into a detection module in the moving target detection model to output a moving target detection result corresponding to the moving target, the detection module being configured to detect the background image and the current frame image.

[0008] In a second aspect, the embodiments of the present application provide a moving target detection model training method, which comprises:

[0009] Determining a category of a moving target;

[0010] Obtaining a sample moving video data set containing the moving target of the category, and labeling the moving target contained in each sample moving video data in the sample moving video data set, the labeling information comprising at least one of labeled center point coordinate information, labeled size length-width information and labeled category information of the moving target;

[0011] Training an initial moving target detection model based on the sample moving video data set with labeled labeling information to obtain a trained moving target detection model.

[0012] In a third aspect, an embodiment of the present application provides a moving target detection device, the moving target detection device comprising:

[0013] a video stream acquisition unit configured to acquire a moving video stream containing a moving target, the moving video stream comprising a current frame image of a current moment and a historical frame image before the current moment;

[0014] a background image acquisition unit configured to input the historical frame image into a background modeling module in a moving target detection model, perform background modeling on the historical frame image based on the background modeling module, and output a background image;

[0015] a moving target detection unit configured to input the background image and the current frame image into a detection module in the moving target detection model, and output a moving target detection result corresponding to the moving target, the detection module being configured to detect the background image and the current frame image.

[0016] In a fourth aspect, an embodiment of the present application provides a moving target detection model training device, the device comprising:

[0017] a category determination unit configured to determine a category of a moving target;

[0018] a sample data acquisition unit configured to acquire a sample moving video data set containing the moving target of the category, and label the moving target contained in each sample moving video data in the sample moving video data set with label information, the label information comprising at least one of label center point coordinate information, label size length-width information, and label category information of the moving target;

[0019] a model training unit configured to train an initial moving target detection model based on the sample moving video data set with the label information labeled, to obtain a trained moving target detection model.

[0020] In a fifth aspect, an embodiment of the present application provides an electronic device, which can include a processor and a memory; wherein the memory stores at least one instruction, the at least one instruction being adapted to be loaded and executed by the processor to perform the method steps described above.

[0021] The technical solutions provided by some embodiments of the present application have at least the following beneficial effects:

[0022] The mobile target detection method provided in the embodiment of the application comprises the following steps: acquiring a mobile video stream containing a mobile target, the mobile video stream comprising a current frame image at a current moment and a historical frame image before the current moment; inputting the historical frame image into a background modeling module in a mobile target detection model to obtain a background image; and inputting the background image and the current frame image into a detection module in the mobile target detection model to output a mobile target detection result corresponding to the mobile target. The detection module is used for detecting the background image and the current frame image, and accurate detection of the mobile target in the mobile video stream is realized. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0024] Figure 1 A system architecture diagram of a mobile target detection method provided in the embodiment of the present application;

[0025] Figure 2 A flowchart of a mobile target detection method provided in the embodiment of the present application;

[0026] Figure 3 A flowchart of a mobile target detection method provided in the embodiment of the present application;

[0027] Figure 4 An exemplary flowchart of a mobile target detection method provided in the embodiment of the present application;

[0028] Figure 5 A flowchart of a mobile target detection model training method provided in the embodiment of the present application;

[0029] Figure 6 A flowchart of a mobile target detection model training method provided in the embodiment of the present application;

[0030] Figure 7 A structural schematic diagram of a mobile target detection device provided in the embodiment of the present application;

[0031] Figure 8 A structural schematic diagram of a mobile target detection device provided in the embodiment of the present application;

[0032] Figure 9 A structural schematic diagram of a mobile target detection model training device provided in the embodiment of the present application;

[0033] Figure 10 A structural diagram of a model training unit provided by an embodiment of the present application is provided.

[0034] Figure 11 A structural block diagram of an electronic device provided by an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0036] In the description of the present application, it should be understood that the terms "first", "second" and the like are used only for the purpose of description, and should not be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units not listed, or optionally includes other steps or units inherent to the process, method, product or device. Those of ordinary skill in the art can understand the specific meaning of the above terms in the present application according to the specific circumstances. In addition, in the description of the present application, "multiple" means two or more, unless otherwise specified. "And / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects.

[0037] Video moving target detection intelligent monitoring, human-computer interaction, behavior analysis, intrusion detection, anomaly detection, object tracking and automatic driving have wide applications. Moving target detection aims to locate and classify the specified moving targets in the video. Traditional moving target detection algorithms are mostly based on traditional background subtraction or optical flow methods, which can achieve certain results in some specific situations, but are difficult to adapt to complex and variable scenes, and the algorithms are not robust. In addition, the implementation mechanism of traditional background modeling methods based on background subtraction and the like depends on many super parameters, which often need to be set by experts with rich experience in the field to obtain reasonable parameter estimation, so it is not practical for actual application scenarios.

[0038] Based on this, this application proposes a moving target detection method. By acquiring a moving video stream containing a moving target, the historical frame images before the current moment in the moving video stream are input into the background modeling module of the moving target detection model. The background modeling module performs background modeling on the historical frame images to obtain a background image. Then, the detection module in the moving target detection model predicts the background image and the current frame image to obtain the moving target detection result corresponding to the moving target. This achieves accurate detection of moving targets in the moving video stream. The deep learning-based moving target detection model has excellent robustness for moving target detection in most scenarios.

[0039] Please see Figure 1 This is a system architecture diagram of a moving target detection method provided in an embodiment of this application. Figure 1 As shown, the system architecture may include a video recording device 101, a network 102, and a cloud 103. The network 102 may include various types of wireless communication links, such as Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links.

[0040] The video recording device 101 can interact with the cloud 103 via network 102. The video recording device 101 can be various electronic devices, including but not limited to wireless cameras, smartwatches, smartphones, tablets, laptops, and desktop computers. The cloud 103 can be a business server providing various services. It should be noted that the cloud 103 can be hardware or software. When the cloud 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the cloud 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0041] In one embodiment, the video shooting device 101 collects a moving video stream containing a moving target, and sends the collected moving video stream to the cloud 103 through the network 102. The cloud includes a trained moving target detection model. After receiving the moving video stream data, the cloud performs moving target detection on the moving video stream by the moving target detection model. Specifically, the moving video stream containing the moving target is obtained, the historical frame image before the current time in the moving video stream is input into the background modeling module in the moving target detection model, the background modeling module is used to model the background of the historical frame image to obtain a background image, and then the detection module in the moving target detection model is used to predict the background image and the current frame image to obtain a moving target detection result corresponding to the moving target. Thus, accurate detection of the moving target in the moving video stream is achieved. The moving target detection model based on deep learning has excellent robustness for moving target detection in most scenes.

[0042] According to Figure 1 The system architecture is shown in the following detailed description of specific embodiments. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. The flowchart shown in the drawings is only an exemplary illustration, and it is not necessary to perform the steps as shown. For example, some steps are parallel and there is no strict logical sequence, so the actual execution order is variable.

[0043] Please refer to Figure 2 , a flowchart of a moving target detection method provided by the embodiment of the present application. In specific embodiments, the moving target detection method can be applied to a moving target detection device, which can be an electronic device or a moving target detection program in an electronic device. In the following, the specific process of the embodiment will be described taking an electronic device as an example. It should be understood that the electronic device to which the embodiment is applied can be a smartphone, a tablet computer, a desktop computer, a wearable device, a wireless camera, etc., which is not limited here. The following will be described in detail with reference to the flowchart shown in Figure 1 The moving target detection method can specifically include the following steps:

[0044] S102, obtaining a moving video stream containing a moving target, the moving video stream including a current frame image at a current time and a historical frame image before the current time;

[0045] The moving video stream refers to video data containing a moving target, which can be a video stream being recorded by a running video shooting device.

[0046] The current frame image refers to a video frame image collected by a video shooting device at a current moment.

[0047] The historical frame image refers to a series of video frame images collected by the video shooting device before the current moment, and the historical frame image can cache video data.

[0048] Specifically, the video shooting device uploads a moving video stream shot by the video shooting device to a mobile target detection model in the cloud, and the mobile target detection model acquires the moving video stream.

[0049] The mobile target detection model is a deep learning model based on a convolutional neural network.

[0050] S104, input the historical frame image into a background modeling module in the mobile target detection model, and perform background modeling on the historical frame image based on the background modeling module and output a background image;

[0051] The background modeling module is used for extracting background information from the historical frame image in the moving video stream and outputting a background image.

[0052] Specifically, after the mobile target detection model acquires the moving video stream, the historical frame image contained in the moving video stream is input into the background modeling module of the mobile target detection model. The background modeling module performs average pooling operation on the historical frame image based on pooling kernels with different receptive fields, obtains at least two feature maps, and then performs fusion processing on the at least two feature maps to obtain a background image.

[0053] The feature map is an image containing background feature information obtained by the background modeling module through background feature extraction on the historical frame image.

[0054] The receptive field refers to the size of the region mapped on the input picture by the pixel points on the feature map output by the feature extraction operator of each layer of the convolutional neural network. It is not difficult to understand that by using at least two pooling kernels with different receptive fields to perform average pooling operation on the historical frame image, at least two feature maps are obtained, and fusion processing is performed on the at least two feature maps to generate a background image. In this way, a feature map with a multi-scale receptive field can be obtained, which improves the accuracy of background information extraction and ensures the precision of background information in the background image.

[0055] Optionally, in an embodiment, after at least two pooling kernels with different receptive fields are used to perform average pooling operation on the historical frame image to obtain at least two feature maps, the feature weights corresponding to each feature map are calculated in the channel dimension, and each feature map is fused according to the feature weights corresponding to each feature map to obtain a background image containing feature weight information of the feature map.

[0056] Optionally, in an embodiment, after the at least two feature maps are obtained by performing the average pooling operation on the historical frame image using the at least two pooling kernels with different receptive fields, the at least two feature maps are encoded and compressed to obtain a fused feature map, then a feature weight of the feature map with different receptive fields in the fused feature map is calculated, the feature weight is added to the fused feature map by feature re-labeling to obtain a weight feature map, and finally the fused feature map and the weight feature map are added to obtain the background image.

[0057] It can be understood that, by calculating the feature weight corresponding to the feature map with different receptive fields and adding the feature weight corresponding to the feature map to the background image, the importance of the key background features in the background image can be improved, and the background information can be more accurate.

[0058] S106, input the background image and the current frame image into a detection module in the moving target detection model, and output a moving target detection result corresponding to the moving target.

[0059] The detection module is configured to predict the background image and the current frame image to obtain the moving target detection result of the moving target in the moving video stream.

[0060] Specifically, after the background modeling module outputs the background image after extracting the background information, the background image and the current frame image at the current time are input into the detection module, and the detection module predicts the background image and the current frame image to obtain the moving target detection result of the moving target in the moving video stream.

[0061] The detection module can be a yolov5 basic network, and the yolov5 basic network can include a ghost-based bottleneck structure and a depth separable convolution module.

[0062] In one embodiment, the moving target detection result includes category information, center point coordinate information and size length-width information, the detection module can include a backbone part, a neck part and three detection heads, the three detection heads are respectively a first detection branch, a second detection branch and a third detection branch. The detection module detects the background image and the current frame image, specifically, the backbone part and the neck part in the detection module extract and encode features of the background image and the current frame image to obtain a deep feature map, the first detection branch detects the deep feature map to determine whether the deep feature map contains a moving target, if it is determined that the deep feature map contains a moving target, the second detection branch in the detection module is used to regress and predict the deep feature map to obtain the center point coordinate information and the size length-width information of the moving target, and the third detection branch in the detection module is used to predict the category of the moving target.

[0063] In one embodiment, the historical frame image before the current moment can also be subjected to median filtering processing in the channel dimension to obtain a median image, the median image, the background image and the current frame image are input into the detection module in the moving target detection model, and the detection module detects the median image, the background image and the current frame image to obtain the moving target detection result corresponding to the moving target.

[0064] It can be understood that the median image is the output of the median filtering of the historical frame image, and contains the background information in the historical frame image. The median image, the background image and the current frame image are input into the detection module, the background information in the background image can be further enhanced, and the detection accuracy is improved.

[0065] In one embodiment, after the background image and the current frame image are input into the detection module in the moving target detection model and the moving target detection result corresponding to the moving target is output, the moving target detection result is synchronized to the mobile terminal, so that the user can view the moving target detection result on the mobile terminal.

[0066] By using the moving target detection method provided in the embodiments of the present application, the historical frame image before the current moment in the moving video stream containing the moving target is input into the background modeling module in the moving target detection model, the background modeling module is used to model the background of the historical frame image to obtain a background image, and then the detection module in the moving target detection model is used to predict the background image and the current frame image to obtain the moving target detection result corresponding to the moving target, so that the moving target in the moving video stream is accurately detected, and the moving target detection model based on deep learning has excellent robustness for moving target detection in most scenes.

[0067] Please refer to Figure 3 A flowchart of a mobile target detection method provided by an embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the mobile target detection method can include the following steps: Figure 3

[0068] S202, acquiring a mobile video stream containing a mobile target, the mobile video stream including a current frame image of a current time and a historical frame image before the current time;

[0069] The mobile video stream refers to video data containing a mobile target, which can be a video stream being recorded by a video shooting device that is running.

[0070] The current frame image refers to a video frame image collected by a video shooting device at a current time.

[0071] The historical frame image refers to a series of video frame images collected by a video shooting device before a current time, and the historical frame image can cache video data. The historical frame image is used to extract background information to obtain a background image.

[0072] Specifically, the video shooting device uploads the mobile video stream shot by the video shooting device to a mobile target detection model in the cloud, and the mobile target detection model acquires the mobile video stream.

[0073] S204, performing average pooling operation on the historical frame image by using pooling kernels with different receptive fields to obtain at least two feature maps;

[0074] S206, encoding and compressing the at least two feature maps to obtain a fusion feature map;

[0075] Specifically, after the mobile target detection model acquires the mobile video stream, the historical frame image contained in the mobile video stream is input to a background modeling module of the mobile target detection model. The background modeling module performs average pooling operation on the historical frame image by using pooling kernels with different receptive fields to obtain at least two feature maps, and then performs fusion processing on the at least two feature maps to obtain a fusion feature map fusing features of different receptive fields.

[0076] The feature map is an image containing background feature information obtained by the background modeling module by performing background feature extraction on the historical frame image.

[0077] ​The receptive field refers to the size of the region on the input picture that is mapped by a pixel point on the feature map output by a feature extraction operator of each layer of the convolutional neural network. It is not difficult to understand that, by adopting the manner of performing average pooling operation on the historical frame image by using at least two pooling kernels with different receptive fields, obtaining at least two feature maps, and performing fusion processing on the at least two feature maps to generate a fusion feature map, a feature map with a multi-scale receptive field can be obtained, and the accuracy of background information extraction can be improved, and the precision of the background information in the background image can be ensured.

[0078] In S208, a feature weight of the feature map with different receptive fields in the fusion feature map is calculated, and the feature weight is added to the fusion feature map by feature re-labeling to obtain a weight feature map.

[0079] In S210, the fusion feature map and the weight feature map are added to obtain the background image.

[0080] Specifically, after the average pooling operation is performed on the historical frame image by using at least two pooling kernels with different receptive fields to obtain at least two feature maps, the at least two feature maps are encoded and compressed to obtain a fusion feature map, then a feature weight of the feature map with different receptive fields in the fusion feature map is calculated, and the feature weight is added to the fusion feature map by feature re-labeling to obtain a weight feature map, and finally the fusion feature map and the weight feature map are added to obtain the background image.

[0081] It can be understood that, by calculating the feature weight corresponding to the feature map with different receptive fields and adding the feature weight information corresponding to the feature map to the background image, the importance of the key background features in the background image can be improved, and the background information can be more accurate.

[0082] In S212, deep feature maps are obtained by performing feature extraction and encoding on the background image and the current frame image based on the detection module.

[0083] Specifically, after the background image is obtained, the background image and the current frame image are input into the detection module in the moving target detection model, and the background image and the current frame image are subjected to feature extraction and encoding by the backbone part and the neck part in the detection module to obtain deep feature maps.

[0084] The backbone is a main network used for extracting features, and the neck is used to improve the diversity and robustness of the features.

[0085] In S214, the deep feature maps are subjected to moving target detection by a first detection branch in the detection module, and it is determined whether the deep feature maps contain a moving target.

[0086] Specifically, the first detection branch is a detection head in the detection module, which is used to detect whether there is a moving target in the deep feature map. If it is determined that there is a moving target in the deep feature map, steps S216 and S218 are executed, and a moving target detection result is output. If no moving target is detected in the deep feature map, the detection module does not output a moving target detection result, or outputs a prompt information that no moving target is detected.

[0087] S216, if it is determined that there is a moving target in the deep feature map, the second detection branch in the detection module is used to perform regression prediction on the deep feature map to obtain center point coordinate information and size length-width information of the moving target.

[0088] Specifically, the second detection branch is used to perform convolution operation on the deep feature map to predict the center point coordinate information and the size length-width information of the moving target.

[0089] S218, if it is determined that there is a moving target in the deep feature map, the third detection branch in the detection module is used to perform category prediction on the deep feature map to obtain category information of the moving target.

[0090] Specifically, the third detection branch is used to perform full connection operation on the deep feature map to predict the category information of the moving target.

[0091] Please refer to Figure 4 , which is an example flow chart of moving target detection provided by the embodiment of the present application.

[0092] As shown in Figure 4 , the n-t frame to the n frame are historical frame images before the current time, the historical frame images are input into the background modeling module to obtain a background image, and the historical frame images from the n-t frame to the n frame are subjected to median filtering processing to obtain a median frame image. The background image, the median frame image and a current frame image at the current time are input into the trained moving target detection model, and finally the moving target detection result of the moving target is obtained by the moving target detection model.

[0093] For example, the moving target detected by the trained moving target detection model is a pedestrian, and the moving video stream is a video picture captured by a fixed camera in real time. After the moving video stream is input into the moving target detection model, the moving target detection model extracts background information of the picture captured by the fixed camera except the pedestrian according to the historical frame images before the current time captured by the fixed camera, generates a background image, and then detects whether there is a pedestrian in the current frame image and the position of the pedestrian by performing pedestrian detection on the background image, the median image and the current frame image.

[0094] For example, the moving target detected by the trained moving target detection model includes pedestrians and vehicles, and the moving video stream is a video captured by a fixed camera in real time. After the moving video stream is input into the moving target detection model, the moving target detection model extracts background information of the video captured by the fixed camera from historical frame images before the current time, generates a background image, and then detects whether there are pedestrians and vehicles in the current frame image and the positions of the pedestrians and vehicles by performing moving target detection on the background image, the median image and the current frame image. In an embodiment, the historical frame images before the current time can also be median filtered in the channel dimension to obtain a median image. The median image, the background image and the current frame image are input into the detection module of the moving target detection model, and the detection module detects the median image, the background image and the current frame image to obtain the moving target detection result corresponding to the moving target.

[0095] The moving target detection method provided by the embodiment of the present application can obtain a moving video stream containing a moving target, input historical frame images before the current time in the moving video stream into a background modeling module of a moving target detection model, perform average pooling operation on the historical frame images by using at least two pooling kernels with different receptive fields to obtain at least two feature maps, fuse the at least two feature maps to obtain a background image, and then perform prediction on the background image and the current frame image by using a detection module in the moving target detection model to obtain a moving target detection result corresponding to the moving target. The method can accurately detect the moving target in the moving video stream. By using at least two pooling kernels with different receptive fields to perform average pooling operation on the historical frame images to obtain at least two feature maps and fuse the at least two feature maps to generate a background image, the accuracy of background information extraction is ensured, and the detection accuracy of the moving target is ensured.

[0096] In an embodiment, before performing moving target detection on the moving video stream recorded by the video capturing device, the moving target detection model needs to be trained. Please refer to Figure 5 The flowchart of a moving target detection model training method provided by the embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the method can include the following steps: Figure 5

[0097] S302, determine the category of the moving target;

[0098] ​Firstly, the category of the moving target to be detected by the moving target detection model is determined, which can be any movable object or biological category. Only after the category of the moving target is determined, the corresponding moving video data can be selected to train the moving target detection model according to the category of the moving target. After the moving target detection model is trained, the moving target detection model can detect the moving target of the corresponding category.

[0099] The category of the moving target can be one or more.

[0100] In S304, a set of sample moving video data containing the moving target is obtained, and the moving target contained in each sample moving video data in the set of sample moving video data is labeled with label information, the label information including at least one of label center point coordinate information, label size length-width information and label category information of the moving target.

[0101] Specifically, after the category of the moving target is determined, a set of sample moving video data of the moving target detection model containing the moving target of the corresponding category is obtained, and the moving target of the corresponding category contained in each sample moving video data is labeled, the content of the labeling including at least one of label center point coordinate information, label size length-width information and label category information of the moving target.

[0102] The sample moving video data is used to train the moving target detection model. Before the sample moving video data is used to train the moving target detection model, the moving target in the sample moving video data is first labeled with information, including but not limited to label center point coordinate information, label size length-width information and label category information of the moving target.

[0103] Optionally, obtaining the set of sample moving video data containing the moving target of the category can be: collecting the set of sample moving video data containing the moving target of the corresponding category in the Internet through a crawler technology.

[0104] Optionally, when the category of the moving target is multiple, in an embodiment, after the set of sample moving video data containing the moving target of the corresponding category in the Internet is collected through the crawler technology, the number of each sample moving video data in the obtained set of sample moving video data can be counted according to the category of the moving target. When it is found that the data amount of the sample moving video data containing a certain category of moving target is small, the number of sample moving video data of the category with small data amount can be expanded by manual collection in different environments, distances and shooting angles.

[0105] Optionally, in an embodiment, after the set of sample moving video data containing the moving target of the corresponding category in the Internet is collected through the crawler technology, the obtained each sample moving video data is processed by data enhancement. The data enhancement processing can include color space transformation, visit transformation, etc.

[0106] S306, based on the labeling, the sample mobile video data set with labeled information is used to train an initial mobile target detection model, and a trained mobile target detection model is obtained;

[0107] Specifically, after labeling information is labeled in each sample mobile video data in the obtained sample mobile video data set, each sample mobile video data is input into the initial mobile target detection model for prediction. The initial mobile target detection model outputs a prediction result of the mobile target in the sample mobile video data. According to the difference between the prediction result output by the initial mobile target detection model and the pre-labeled labeling information, the model parameters of the initial mobile target detection model are adjusted to realize the training process of the initial mobile target detection model, and finally a trained mobile target detection model is obtained.

[0108] It can be understood that the labeling information is the real position and category information of the mobile target in the sample mobile video, and the prediction result output by the initial mobile target detection model is the predicted position and category information of the mobile target in the sample mobile video. There is a difference between the prediction result output by the initial mobile target detection model and the pre-labeled labeling information. The process of training the model is to reduce the difference between the prediction result and the labeling information.

[0109] In one embodiment, referring to Figure 6 , step S306 can include the following steps:

[0110] S3061, extracting the sample historical frame image and the sample current frame image corresponding to each sample mobile video data in the sample mobile video data set, respectively;

[0111] Specifically, the sample mobile video data is a video data containing a mobile target of a specified category that has been recorded. In the sample mobile video data, a sample historical frame image for extracting background information and a sample current frame image for detecting a mobile target are determined.

[0112] For example, a sample mobile video data contains 200 frames of images. The first 150 frames of images can be used as sample historical frame images for extracting background information, and the 151st frame of image can be used as a sample current frame image for detecting a mobile target.

[0113] S3062, inputting the sample historical frame image into a background modeling module in the initial mobile target detection model to output a sample background image;

[0114] Specifically, the initial mobile target detection model includes a background modeling module. The background modeling module is used to extract background information in the sample mobile video data except the mobile target according to the sample historical frame image, and generate a sample background image containing the background information.

[0115] In one embodiment, the background modeling module performs an average pooling operation on the sample historical frame image by using at least two pooling kernels with different receptive fields to obtain at least two sample feature maps, then performs encoding compression on the at least two sample feature maps to obtain a sample fusion feature map, calculates feature weights of the sample feature maps with different receptive fields in the sample fusion feature map, and finally adds the feature weights to the sample fusion feature map by feature recalibration to obtain a sample weight feature map, and adds the sample fusion feature map and the sample weight feature map to obtain a sample background image.

[0116] The sample feature map is an image containing background feature information obtained by the background modeling module by performing background feature extraction on the sample historical frame image.

[0117] The receptive field refers to the size of the region on the input picture mapped by the pixel points on the feature map output by the feature extraction operator of each layer of the convolutional neural network. It is not difficult to understand that by performing an average pooling operation on the sample historical frame image by using at least two pooling kernels with different receptive fields to obtain at least two sample feature maps, and performing fusion processing on the at least two sample feature maps to generate a sample background image, a feature map of a sample with a multi-scale receptive field can be obtained, which improves the accuracy of background information extraction and ensures the precision of the background information in the background image.

[0118] S3063, inputting the sample background image and the sample current frame image into the detection module in the initial moving target detection model to output a sample moving target detection result corresponding to the moving target;

[0119] Specifically, the initial moving target detection model includes a detection module, which is used to detect the moving target in the sample moving video data according to the sample background image and the sample current frame image. It is not difficult to understand that the sample background image contains background information other than the moving target in the sample moving video data, and the current frame image contains background information and the moving target. By comparing the sample background image and the sample current frame image, the detection of the moving target can be completed.

[0120] In one embodiment, the detection module includes a backbone part, a neck part, and three detection heads, i.e., a first detection branch, a second detection branch, and a third detection branch. After the sample background image and the sample current frame image are input into the detection model, the backbone part and the neck part of the detection module perform feature extraction and encoding on the sample background image and the sample current frame image to obtain a sample deep feature map. The first detection branch detects the sample deep feature map to determine whether there is a moving target in the sample deep feature map. If it is determined that there is a moving target in the sample deep feature map, the second detection branch of the detection module is used to perform regression prediction on the sample deep feature map to obtain the center point coordinate information and the size length-width information of the moving target. The third detection branch of the detection module is used to perform category prediction on the sample deep feature map to obtain the category information of the moving target.

[0121] Optionally, the second detection branch performs regression prediction on the deep feature map based on a convolution operation to predict the center point coordinate information and the size length-width information of the moving target. The third detection branch performs category prediction on the deep feature map based on a full connection operation to predict the category information of the moving target.

[0122] It can be understood that, in a convolutional neural network, the convolution operation with translational equivariance is more suitable for processing regression prediction tasks, and the full connection operation can better process classification prediction tasks. Therefore, the use of the convolution operation for regression prediction and the use of the full connection operation for classification prediction can greatly improve the prediction accuracy.

[0123] In one embodiment, the sample historical frame image can also be subjected to median filtering in the channel dimension to obtain a sample median image. The sample median image, the sample background image, and the sample current frame image are input into the detection module in the initial moving target detection model, and the detection module detects the sample median image, the sample background image, and the sample current frame image to obtain the moving target detection result corresponding to the moving target.

[0124] It can be understood that the sample median image is the output of the sample historical frame image after median filtering, and it contains the background information in the sample historical frame image. Inputting the sample median image, the sample background image, and the sample current frame image into the detection module can further enhance the background information in the sample background image, thereby improving the detection accuracy and the training effect of the initial moving target detection model.

[0125] S3064, constructing a cost function based on the sample moving target detection result and the annotation information corresponding to the sample moving video data;

[0126] S3065, train the initial moving target detection model based on the cost function to obtain the trained moving target detection model.

[0127] The moving target detection model trained using the moving target detection model training method provided in the embodiments of this application can perform the following: Figure 2 and Figure 3 The method described involves acquiring a moving video stream containing a moving target, inputting historical frame images from the moving video stream up to the current time point into the background modeling module of the moving target detection model, and then performing average pooling on the historical frame images using at least two pooling kernels with different receptive fields to obtain at least two feature maps. These at least two feature maps are then fused to obtain a background image. Finally, the detection module in the moving target detection model predicts the background image and the current frame image to obtain the moving target detection result. This achieves accurate detection of moving targets in the moving video stream. By using at least two pooling kernels with different receptive fields to perform average pooling on the historical frame images to obtain at least two feature maps, and then fusing these at least two feature maps to generate the background image, the accuracy of background information extraction is ensured, thereby guaranteeing the detection accuracy of the moving target.

[0128] Please see Figure 7 This is a schematic diagram of the structure of a moving target detection device provided in an embodiment of this application. Figure 7 As shown, the moving target detection device 1 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the moving target detection device includes a main control chip and a communication chip. The moving target detection device 1 also includes a video stream acquisition unit 11, a background image acquisition unit 12, and a moving target detection unit 13, specifically including:

[0129] The video stream acquisition unit 11 is used to acquire a moving video stream containing a moving target, wherein the moving video stream includes the current frame image at the current moment and historical frame images before the current moment;

[0130] Background image acquisition unit 12 is used to input the historical frame image into the background modeling module in the moving target detection model, perform background modeling on the historical frame image based on the background modeling module, and output the background image.

[0131] The moving target detection unit 13 is used to input the background image and the current frame image into the detection module in the moving target detection model and output the moving target detection result corresponding to the moving target. The detection module is used to detect the background image and the current frame image.

[0132] Optionally, the background image acquisition unit 12 is specifically used for:

[0133] performing average pooling operation on the historical frame image by using pooling kernels with different receptive fields to obtain at least two feature maps;

[0134] performing encoding compression on the at least two feature maps to obtain a fusion feature map;

[0135] calculating feature weights of feature maps with different receptive fields in the fusion feature map, and adding the feature weights to the fusion feature map by feature re-labeling to obtain a weight feature map;

[0136] adding the fusion feature map and the weight feature map to obtain a background image.

[0137] Optionally, the moving target detection result includes category information, center point coordinate information and size length-width information, and the moving target detection unit 13 is specifically used for:

[0138] performing feature extraction and encoding on the background image and the current frame image based on the detection module to obtain a deep feature map;

[0139] performing moving target detection on the deep feature map based on a first detection branch in the detection module to determine whether the deep feature map contains a moving target;

[0140] if it is determined that the deep feature map contains a moving target, performing regression prediction on the deep feature map based on a second detection branch in the detection module to obtain center point coordinate information and size length-width information of the moving target;

[0141] if it is determined that the deep feature map contains a moving target, performing category prediction on the deep feature map based on a third detection branch in the detection module to obtain category information of the moving target.

[0142] Optionally, the moving target detection unit 13 is further used for:

[0143] performing median filtering processing on the historical frame image in the channel dimension to output a median image;

[0144] inputting the background image and the current frame image into a detection module in a moving target detection model to output a moving target detection result corresponding to the moving target, including:

[0145] The median image, the background image, and the current frame image are input into the detection module of the moving target detection model, and the moving target detection result corresponding to the moving target is output. The detection module is used to detect the median image, the background image, and the current frame image.

[0146] Optional, please see Figure 8 This is a schematic diagram of the structure of a moving target detection device provided in an embodiment of this application. Figure 8 As shown, the moving target detection device further includes:

[0147] The detection result synchronization unit 14 is used to synchronize the detection result of the moving target to the mobile terminal.

[0148] The moving target detection method provided in this application acquires a moving video stream containing a moving target. Historical frame images from the moving video stream up to the current time are input into the background modeling module of the moving target detection model. The background modeling module performs average pooling on the historical frame images using at least two pooling kernels with different receptive fields to obtain at least two feature maps. These at least two feature maps are then fused to obtain a background image. The detection module in the moving target detection model then predicts the background image and the current frame image to obtain the moving target detection result corresponding to the moving target. This achieves accurate detection of moving targets in the moving video stream. By using at least two pooling kernels with different receptive fields to perform average pooling on the historical frame images to obtain at least two feature maps, and then fusing these at least two feature maps to generate a background image, the accuracy of background information extraction is ensured, thereby guaranteeing the detection accuracy of the moving target.

[0149] Please see Figure 9 This is a schematic diagram of the structure of a moving target detection model training device provided in an embodiment of this application. Figure 9 As shown, the moving target detection model training device 2 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the moving target detection model training device includes a main control chip and a communication chip. The moving target detection model training device 2 also includes a category determination unit 21, a sample data acquisition unit 22, and a model training unit 23, specifically including:

[0150] Category determination unit 21 is used to determine the category of the moving target;

[0151] The sample data acquisition unit 22 is configured to acquire a sample moving video data set containing the moving target of the category, and mark the moving target contained in each sample moving video data in the sample moving video data set with marking information, wherein the marking information comprises at least one of marking center point coordinate information, marking size length-width information and marking category information of the moving target.

[0152] The model training unit 23 is configured to train an initial moving target detection model based on the sample moving video data set with the marking information completed, to obtain a trained moving target detection model.

[0153] Optionally, please refer to Figure 10 FIG. 3 is a structural schematic diagram of a model training unit provided by an embodiment of the present application. As shown in FIG. 3, the model training unit 23 comprises: Figure 10

[0154] The image extraction subunit 231 is configured to extract a sample historical frame image and a sample current frame image corresponding to each sample moving video data in the sample moving video data set, respectively.

[0155] The background extraction subunit 232 is configured to input the sample historical frame image into a background modeling module in the initial moving target detection model, to output a sample background image.

[0156] The result prediction subunit 233 is configured to input the sample background image and the sample current frame image into a detection module in the initial moving target detection model, to output a sample moving target detection result corresponding to the moving target.

[0157] The function construction subunit 234 is configured to construct a cost function based on the sample moving target detection result and the marking information corresponding to the sample moving video data.

[0158] The model training subunit 235 is configured to train the initial moving target detection model based on the cost function, to obtain a trained moving target detection model.

[0159] Optionally, the background extraction subunit 232 is specifically configured to:

[0160] Perform an average pooling operation on the sample historical frame image with a pooling kernel having different receptive fields to obtain at least two sample feature maps;

[0161] Encode and compress the at least two sample feature maps to obtain a sample fusion feature map;

[0162] ​Calculate feature weights of sample feature maps of different receptive fields in the sample fusion feature map, and add the feature weights to the sample fusion feature map by feature recalibration to obtain a sample weight feature map;

[0163] Add the sample fusion feature map and the sample weight feature map to obtain a sample background image.

[0164] Optionally, the sample moving target detection result includes sample category information, sample center point coordinate information and sample size length-width information, and the result prediction subunit 233 is specifically configured to:

[0165] Based on the detection module, feature extraction and encoding are performed on the sample background image and the sample current frame image to obtain a sample deep feature map;

[0166] Based on the first detection branch in the detection module, moving target detection is performed on the sample deep feature map to determine whether the sample deep feature map contains a moving target;

[0167] If it is determined that the sample deep feature map contains a moving target, then based on the second detection branch in the detection module, regression prediction is performed on the sample deep feature map to obtain center point coordinate information and size length-width information of the moving target;

[0168] If it is determined that the sample deep feature map contains a moving target, then based on the third detection branch in the detection module, category prediction is performed on the sample deep feature map to obtain category information of the moving target.

[0169] Optionally, the result prediction subunit 233 is further configured to:

[0170] Perform median filtering on the sample historical frame image to output a sample median image;

[0171] Input the sample median image, the sample background image and the sample current frame image into the detection module in the initial moving target detection model to output a sample moving target detection result corresponding to the moving target.

[0172] The moving target detection model trained by the moving target detection model training method provided in the embodiments of the present application can perform the following operations Figure 2 and Figure 3The method described involves acquiring a moving video stream containing a moving target, inputting historical frame images from the moving video stream up to the current time point into the background modeling module of the moving target detection model, and then performing average pooling on the historical frame images using at least two pooling kernels with different receptive fields to obtain at least two feature maps. These at least two feature maps are then fused to obtain a background image. Finally, the detection module in the moving target detection model predicts the background image and the current frame image to obtain the moving target detection result. This achieves accurate detection of moving targets in the moving video stream. By using at least two pooling kernels with different receptive fields to perform average pooling on the historical frame images to obtain at least two feature maps, and then fusing these at least two feature maps to generate the background image, the accuracy of background information extraction is ensured, thereby guaranteeing the detection accuracy of the moving target.

[0173] This application also provides a computer storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1 to 6 The moving target detection method described in this embodiment can be found in the following document for its specific execution process. Figures 1 to 6 The specific details of the embodiments are not elaborated here.

[0174] This application also provides a computer program product that stores at least one instruction, which is loaded and executed by the processor as described above. Figures 1 to 6 The moving target detection method described in this embodiment can be found in the following document for its specific execution process. Figures 1 to 6 The specific details of the embodiments are not elaborated here.

[0175] Please refer to Figure 11 This diagram illustrates a structural block diagram of an electronic device provided in an exemplary embodiment of this application. The electronic device may include one or more components such as a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0176] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the entire electronic device by various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 110 can integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes an operating system, a user interface, and an application program; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but can be implemented by a separate communication chip.

[0177] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). Alternatively, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 can be used to store instructions, programs, codes, code sets or instruction sets.

[0178] Among them, the input device 130 is used to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In the embodiment of the present application, the input device 130 can be a temperature sensor for obtaining the operating temperature of the electronic device. The output device 140 can be a speaker for outputting an audio signal.

[0179] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above-mentioned drawings does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the drawings, or combine certain components, or different component arrangements. For example, the electronic device also includes radio frequency circuitry, an input unit, a sensor, audio circuitry, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module, and the like, which are not described here.

[0180] In the embodiments of the present application, the execution subject of each step can be the electronic device introduced above. Alternatively, the execution subject of each step is the operating system of the electronic device. The operating system can be an Android system, an IOS system, or other operating systems, which are not limited in the embodiments of the present application.

[0181] In the electronic device, Figure 11 In the electronic device, the processor 110 can be configured to invoke the mobile target detection program stored in the memory 120 and execute to implement the mobile target detection method as described in the various method embodiments of the present application.

[0182] By using the mobile target detection method provided in the embodiments of the present application, the mobile video stream containing the mobile target is obtained, the historical frame image before the current time in the mobile video stream is input into the background modeling module in the mobile target detection model, the background modeling module performs average pooling operation on the historical frame image by using at least two pooling kernels with different receptive fields, obtains at least two feature maps, and performs fusion processing on the at least two feature maps to obtain a background image, and then the detection module in the mobile target detection model is used to predict the background image and the current frame image to obtain the mobile target detection result corresponding to the mobile target, which realizes accurate detection of the mobile target in the mobile video stream. By using at least two pooling kernels with different receptive fields to perform average pooling operation on the historical frame image to obtain at least two feature maps, and then performing fusion processing on the at least two feature maps to generate a background image, the accuracy of background information extraction is ensured, and the detection accuracy of the mobile target is ensured.

[0183] Those skilled in the art can clearly understand that the technical solutions of the present application can be implemented by means of software and / or hardware. The "unit" and "module" in the specification refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, and the hardware can be, for example, a field programmable gate array (FPGA), an integrated circuit (IC), and the like.

[0184] It should be noted that, for the foregoing method embodiments, the sequences of the described actions are not necessarily required to achieve the objects of the application, and certain steps can be performed in other sequences or even concurrently. Additionally, the described embodiments are merely provided as examples, and not all of the actions and modules are necessarily required to achieve the objects of the application.

[0185] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0186] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, another division manner can be adopted. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some service interfaces. The coupling or communication connection can be electrical or in other forms.

[0187] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units. That is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0188] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0189] Those skilled in the art can understand that all or part of the steps in the above embodiments of various methods can be completed by a program instructing related hardware. The program can be stored in a computer readable memory, and the memory can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0190] The above descriptions are only exemplary embodiments of the present application, and cannot limit the scope of the present application. That is, any equivalent changes and modifications made according to the teachings of the present application are still within the scope of the present application. Other embodiments of the present application will be readily apparent to those skilled in the art upon considering the specification and practice of the disclosure herein. The present application is intended to cover any variations, uses, or adaptations of the present application following the general principles thereof and including such departures from the present disclosure as come within known or customary practice in the art to which the present application pertains. The specification and examples are to be regarded as exemplary only, and the scope and spirit of the application are to be determined by the claims.

Claims

1. A moving object detection method characterized by, The method comprises: acquiring a moving video stream containing a moving target, the moving video stream comprising a current frame image of a current moment and a historical frame image before the current moment; wherein the current frame image refers to a video frame image collected by a video shooting device at the current moment, and the historical frame image refers to a series of video frame images collected by the video shooting device before the current moment; inputting the historical frame image into a background modeling module in a moving target detection model, performing average pooling operation on the historical frame image based on the background modeling module using pooling kernels with different receptive fields to obtain at least two feature maps; encoding and compressing the at least two feature maps to obtain a fused feature map; calculating feature weights of the feature maps with different receptive fields in the fused feature map, and adding the feature weights to the fused feature map in a feature recalibration manner to obtain a weight feature map; and adding the fused feature map and the weight feature map to obtain a background image; inputting the background image and the current frame image into a detection module in the moving target detection model, performing feature extraction and encoding on the background image and the current frame image based on the detection module to obtain a deep feature map; performing moving target detection on the deep feature map based on a first detection branch in the detection module to determine whether the deep feature map contains a moving target; if it is determined that the deep feature map contains a moving target, performing regression prediction on the deep feature map based on a second detection branch in the detection module to obtain center point coordinate information and size length-width information of the moving target; and if it is determined that the deep feature map contains a moving target, performing category prediction on the deep feature map based on a third detection branch in the detection module to obtain category information of the moving target.

2. The method of claim 1, wherein, After the adding of the fused feature map and the weight feature map to obtain the background image, the method further comprises: performing median filtering processing on the historical frame image in the channel dimension to output a median image; inputting the median image, the background image and the current frame image into the detection module in the moving target detection model, performing feature extraction and encoding on the median image, the background image and the current frame image based on the detection module to obtain a deep feature map; performing moving target detection on the deep feature map based on the first detection branch in the detection module to determine whether the deep feature map contains a moving target; if it is determined that the deep feature map contains a moving target, performing regression prediction on the deep feature map based on the second detection branch in the detection module to obtain center point coordinate information and size length-width information of the moving target; and if it is determined that the deep feature map contains a moving target, performing category prediction on the deep feature map based on the third detection branch in the detection module to obtain category information of the moving target.

3. The method of claim 1, wherein, The method further comprises: synchronizing the center point coordinate information, the size length-width information and the category information of the moving target to a mobile terminal.

4. A mobile target detection model training method, characterized in that, The method comprises: determining the category of the moving target; Obtain a sample mobile video data set containing the mobile target of the category, and label annotation information contained in each sample mobile video data in the sample mobile video data set, the annotation information including at least one of the labeled center point coordinate information, the labeled size length-width information and the labeled category information of the mobile target; Extract a sample historical frame image and a sample current frame image corresponding to each sample mobile video data in the sample mobile video data set respectively, wherein the sample historical frame image is a plurality of frames of images in the sample mobile video data in time sequence before the sample current frame image; Input the sample historical frame image into a background modeling module of an initial mobile target detection model, obtain at least two sample feature maps by performing average pooling operations on the sample historical frame image based on the background modeling module using pooling kernels with different receptive fields, perform encoding compression on the at least two sample feature maps to obtain a sample fusion feature map, calculate feature weights of the sample feature maps with different receptive fields in the sample fusion feature map, and add the feature weights to the sample fusion feature map by feature recalibration to obtain a sample weight feature map, and add the sample fusion feature map and the sample weight feature map to obtain a sample background image; Input the sample background image and the sample current frame image into a detection module of the initial mobile target detection model, perform feature extraction and encoding on the sample background image and the sample current frame image based on the detection module to obtain a sample deep feature map, perform mobile target detection on the sample deep feature map based on a first detection branch in the detection module to determine whether the sample deep feature map contains a mobile target, perform regression prediction on the sample deep feature map based on a second detection branch in the detection module to obtain center point coordinate information and size length-width information of the mobile target if it is determined that the sample deep feature map contains a mobile target, and perform category prediction on the sample deep feature map based on a third detection branch in the detection module to obtain category information of the mobile target if it is determined that the sample deep feature map contains a mobile target; Construct a cost function based on the center point coordinate information, the size length-width information, the category information of the mobile target and the annotation information corresponding to the sample mobile video data, and train the initial mobile target detection model based on the cost function to obtain a trained mobile target detection model.

5. The method of claim 4, wherein, After obtaining the sample background image by adding the sample fusion feature map and the sample weight feature map, the method further includes: Performing median filtering processing on the sample historical frame image to output a sample median image. The sample median image, the sample background image, and the sample current frame image are input into a detection module in the initial moving target detection model, feature extraction and encoding of the sample median image, the sample background image, and the sample current frame image are performed based on the detection module to obtain a sample deep feature map; moving target detection is performed on the sample deep feature map based on a first detection branch in the detection module, and it is determined whether the sample deep feature map contains a moving target; if it is determined that the sample deep feature map contains a moving target, regression prediction is performed on the sample deep feature map based on a second detection branch in the detection module to obtain center point coordinate information and size length-width information of the moving target; if it is determined that the sample deep feature map contains a moving target, category prediction is performed on the sample deep feature map based on a third detection branch in the detection module to obtain category information of the moving target.

6. The method of claim 4, wherein, The method comprises: The method comprises: The method comprises:

7. A moving object detection apparatus characterized by comprising: The method comprises: The device comprises: A video stream acquisition unit is configured to acquire a moving video stream containing a moving target, the moving video stream comprising a current frame image at a current time and historical frame images before the current time; wherein the current frame image refers to a video frame image captured by a video shooting device at the current time, and the historical frame images refer to a series of video frame images captured by the video shooting device before the current time. A background image acquisition unit is configured to input the historical frame images into a background modeling module in a moving target detection model, perform average pooling operations on the historical frame images by using pooling kernels with different receptive fields based on the background modeling module, and obtain at least two feature maps; encode and compress the at least two feature maps to obtain a fused feature map; calculate feature weights of the feature maps with different receptive fields in the fused feature map, and add the feature weights to the fused feature map by feature recalibration to obtain a weight feature map; and add the fused feature map and the weight feature map to obtain a background image. The mobile target detection unit is configured to input the background image and the current frame image into a detection module in the mobile target detection model, perform feature extraction and encoding on the background image and the current frame image based on the detection module to obtain a deep feature map, perform mobile target detection on the deep feature map based on a first detection branch in the detection module, and determine whether the deep feature map contains a mobile target. If it is determined that the deep feature map contains a mobile target, the second detection branch in the detection module is used to perform regression prediction on the deep feature map to obtain center point coordinate information and size length-width information of the mobile target. If it is determined that the deep feature map contains a mobile target, the third detection branch in the detection module is used to perform category prediction on the deep feature map to obtain category information of the mobile target. 8.A mobile target detection model training apparatus, characterized in that, The device comprises: a category determination unit configured to determine a category of a mobile target; a sample data acquisition unit configured to acquire a sample mobile video data set containing the mobile target of the category, and label the mobile target contained in each sample mobile video data in the sample mobile video data set, the label information including at least one of labeled center point coordinate information, labeled size length-width information and labeled category information of the mobile target; a model training unit configured to extract a sample historical frame image and a sample current frame image corresponding to each sample mobile video data in the sample mobile video data set respectively, wherein the sample historical frame image is a plurality of images in the sample mobile video data that are in time sequence before the sample current frame image; The model training unit is further configured to input the sample historical frame image into a background modeling module of an initial mobile target detection model, perform average pooling operation on the sample historical frame image based on the background modeling module using a pooling kernel with different receptive fields to obtain at least two sample feature maps, perform encoding compression on the at least two sample feature maps to obtain a sample fusion feature map, calculate feature weights of the sample feature maps with different receptive fields in the sample fusion feature map, and add the feature weights to the sample fusion feature map through feature recalibration to obtain a sample weight feature map, and add the sample fusion feature map and the sample weight feature map to obtain a sample background image. The model training unit is further configured to input the sample background image and the sample current frame image into a detection module of the initial moving target detection model, perform feature extraction and encoding on the sample background image and the sample current frame image based on the detection module to obtain a sample deep feature map, perform moving target detection on the sample deep feature map based on a first detection branch in the detection module, and determine whether the sample deep feature map contains a moving target; if it is determined that the sample deep feature map contains a moving target, perform regression prediction on the sample deep feature map based on a second detection branch in the detection module to obtain center point coordinate information and size length-width information of the moving target; and if it is determined that the sample deep feature map contains a moving target, perform category prediction on the sample deep feature map based on a third detection branch in the detection module to obtain category information of the moving target. The model training unit is further configured to construct a cost function based on the center point coordinate information, the size length-width information, the category information of the moving target, and the label information corresponding to the sample moving video data, and train the initial moving target detection model based on the cost function to obtain a trained moving target detection model.

9. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program which is suitable for being loaded and executed by the processor to perform the steps of the method in any one of claims 1-3 or 4-6.

Images