A moving target detection method, medium and device based on semi-supervised learning

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method, medium and device for detecting moving targets based on semi-supervised learning, comprising the following steps: A. acquiring a data set, including an automatically generated set, a semi-manually annotated set, a fully manually annotated set and a test set; B. establishing a neural network parameter model, the neural network parameter model including an image input device, a feature extractor and a structure output device connected in sequence; C. training the neural network parameter model based on the automatically generated set, the semi-manually annotated set and the fully manually annotated set in the acquired data set, and calculating a loss function until the loss function converges to a preset value, so as to obtain a trained neural network parameter model; and D. performing moving target detection on the test set using the trained neural network parameter model, wherein the moving target can be effectively detected when the camera is moving, the moving target features are automatically learned, the moving target is not easily affected by the environment and the algorithm has better robustness.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a moving target detection method based on semi-supervised learning. Background Art

[0002] Motion target detection involves removing long-term, unchanging background information from large video sequences to detect moving objects. This technology has practical applications in smart security, virtual reality, navigation and guidance, traffic control, target tracking, human-computer interaction, and other fields. Currently, object detection methods for video images primarily rely on obtaining the location and category of a few pre-set targets. These methods can only distinguish the location of specific targets within an image; they cannot determine the target's motion state, whether it is stationary or moving. Therefore, they are not suitable for specialized applications such as smart security scenarios like area intrusion detection.

[0003] While some intelligent moving target detection methods already exist, they are essentially model-based. These methods rely on subjective, artificial feature design to establish mathematical models for moving target detection. These methods, for example, rely on inter-frame differences, background modeling, and optical flow. These methods suffer from poor algorithm robustness and generalization. For example, in complex surveillance environments subject to lighting variations, occlusions, camera shake, and shadows, classic intelligent moving target detection methods can suffer from numerous false positives and false negatives. After optimizing for one scenario, the algorithm's performance degrades dramatically when switching to another, resulting in weak generalization and inability to achieve universal and replicable performance. Furthermore, classic intelligent moving target detection methods can only be applied to scenes where the camera is stationary; they fail in situations involving camera motion.

[0004] CN109740563B disclosed a motion target detection method for video surveillance on February 12, 2021. It samples the input RGB video stream, detects and fuses the R channel, G channel and B channel respectively, and can be processed in parallel at high speed on dedicated hardware devices such as FPGA. However, the input image set only comes from video surveillance, and has poor applicability to all scenarios and low robustness. Summary of the Invention

[0005] To address the above problems, the present invention proposes a motion target detection method based on semi-supervised learning. Unlike the traditional intelligent motion target detection method based on background modeling, the present invention uses a deep learning neural network and a large amount of camera static and camera motion data training to automatically learn the motion target features, thereby realizing end-to-end motion target detection and improving the robustness and generalization of the intelligent motion target detection algorithm.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting a moving target based on semi-supervised learning, comprising the steps of:

[0007] A. Obtain a dataset, which includes an automatically generated set, a semi-manually annotated set, a fully manually annotated set, and a test set;

[0008] B. Establish a neural network parameter model, which includes an image input device, a feature extractor, and a structure output device connected in sequence; the neural network parameter model is used to extract high-dimensional abstract features of images.

[0009] C. Training the neural network parameter model based on the automatically generated set, semi-manually annotated set, and fully manually annotated set in the acquired data set, and calculating the loss function until the loss function converges to a preset value to obtain a trained neural network parameter model;

[0010] D. Use the trained neural network parameter model to detect moving targets on the test set.

[0011] In an optional scheme of the first aspect, the image input device includes two first convolution layers and a feature stitching layer arranged in parallel. The two first convolution layers arranged in parallel are used to input the first image and the second image respectively to obtain two feature maps, and the feature stitching layer is used to stitch the two feature maps to obtain a first stitching feature map.

[0012] In another optional scheme of the first aspect, the image input device includes a feature stitching layer and a second convolution layer connected in sequence, the feature stitching layer is used to stitch two preprocessed input images, the preprocessing is to process the input image into a single-channel grayscale image to obtain a stitching image, and the second convolution layer is used to convolve the obtained stitching image to obtain a second stitching feature map.

[0013] In another optional scheme of the first aspect, the feature extractor includes multiple layers of downsampling convolution layers and corresponding multiple layers of upsampling deconvolution layers connected in sequence, and corresponding channel splicing layers are connected between the downsampling convolution layers and the corresponding upsampling deconvolution layers.

[0014] In another optional scheme of the first aspect, the structure outputter includes three parallel convolution layers, corresponding to the three convolution results respectively, the first output represents the target position in the first frame image of the input, the second output represents the target position in the second frame image of the input image, and the third output represents the displacement between the corresponding position targets.

[0015] In another optional solution of the first aspect, in step C, two sequence images separated by a preset number of frames are selected from the acquired data set and scaled to obtain two images with the same pixel size to train the neural network parameter model.

[0016] In another optional scheme of the first aspect, in step C, when training is performed using the automatically generated set, the semi-manually labeled set, and the fully manual labeled set in sequence, the corresponding parameter learning rate is set, the automatically generated set is to randomly generate geometric figures to simulate moving targets, the semi-manually labeled set is to randomly select screenshots as moving targets, and the fully manual labeled set is to manually calibrate the moving targets.

[0017] In another optional solution of the first aspect, in step C, the loss function, denoted as L, is calculated as follows:

[0018] L=λ1L mov +λ2L distance +λ3L loc

[0019] L mov = -[c·log(p)+(1-c)log(1-p)]

[0020]

[0021]

[0022] Among them, L mov Indicates whether the moving target has a loss value, L distance Indicates the displacement loss value of the moving target center, L loc represents the loss value of the predicted position of the moving target, λ1 represents the weight of the loss value of whether the moving target moves, λ2 represents the weight of the displacement loss value of the center of the moving target, λ3 represents the weight of the loss value of the predicted position of the moving target, c represents the true value of whether the moving target moves, p represents the predicted value of whether the moving target moves, Δx represents the true value of the offset of the center of the moving target on the horizontal axis, Δy represents the true value of the offset of the center of the moving target on the vertical axis, Indicates the predicted value of the offset of the moving target center on the horizontal axis, Represents the predicted value of the offset of the moving target center on the ordinate, x represents the true value of the moving target's central abscissa, y represents the true value of the moving target's central ordinate, w represents the true value of the moving target's width, and h represents the true value of the moving target's height. Represents the predicted value of the central horizontal coordinate of the moving target, Represents the predicted value of the central vertical coordinate of the moving target, Indicates the predicted value of the moving target width, Indicates a high predicted value for a moving target.

[0023] In the second aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute a motion target detection method based on semi-supervised learning as provided in the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect.

[0024] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute a motion target detection method based on semi-supervised learning as provided in the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.

[0025] The beneficial technical effects of the present invention include:

[0026] 1. The present invention can effectively distinguish moving targets from backgrounds by extracting high-dimensional abstract features of images, and can effectively detect moving targets when the camera is moving.

[0027] 2. Unlike the existing "model-driven" motion target detection method that uses artificially designed features to model the background of the image to achieve motion target detection, the present invention is based on the "data-driven" deep learning algorithm model. It automatically learns the motion target features through deep convolutional neural networks and massive training data. It can effectively solve the problem that the existing intelligent motion target detection method is easily affected by the environment and has better algorithm robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a flow chart of a moving target detection method based on semi-supervised learning in the present invention;

[0030] Figure 2 This is a block diagram of a neural network parameter model based on semi-supervised learning in the present invention. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0032] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application. Different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, even though the embodiment may not be clearly described in the following text.

[0033] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the elements described without departing from the scope of the present application. Various examples may appropriately omit, replace, or add various processes or components. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted, or combined. In addition, features described in some examples may be combined in other examples.

[0034] Example 1:

[0035] Reference Figure 1 , a motion target detection method based on semi-supervised learning, by establishing a neural network parameter model, and then training the neural network parameter model through an automatically generated set, a semi-manually labeled set, and a fully manually labeled set, to automatically learn the characteristics of the motion target. The parameter training process of the neural network parameter model is divided into three stages. First, pre-training is performed based on the automatically generated set and the semi-manually labeled set. The training process can automatically generate a large amount of training data while training, without manual intervention, to achieve semi-supervised training. Then, based on the real scene data set, that is, the fully manually labeled set, the deep learning neural network is corrected and trained to obtain a trained neural network parameter model. The model takes the video stream and the corresponding camera parameters as input, and directly obtains the position of the moving target in the video through the forward reasoning analysis of the neural network, thereby realizing end-to-end intelligent motion target detection.

[0036] Including steps:

[0037] Step A: Obtain data sets, including automatically generated sets, semi-manually annotated sets, and fully manually annotated sets.

[0038] Automatically generated datasets: A batch of open-source image datasets is collected as background images. First, a random image is selected from the dataset and then preprocessed, including changes to brightness, contrast, and noise. Computer software then randomly generates regular and irregular geometric shapes of varying shapes and pastes them onto the preprocessed background image. The positions of these shapes are then uniformly shifted within the same background image to form an image sequence, simulating the motion of a moving target in a dynamic video.

[0039] The geometric figures in the same image sequence are the same, and the size of the geometric figures can change, with the change range not exceeding 50% of the previous geometric figure size. The number of geometric figures is not less than 5, and the ratio of the area of ​​a single geometric figure to the image area is s, satisfying 0.0002≤s≤0.02. The target position of each frame image and the circumscribed rectangular frame of the geometric figure are automatically generated. Whether the target moves between adjacent images and the displacement of the target (c, Δx, Δy) are recorded, where c represents the true value of whether the target moves, Δx represents the true value of the offset of the center of the moving target on the horizontal axis, and Δy represents the true value of the offset of the center of the moving target on the vertical axis.

[0040] Furthermore, the automatically generated set can be generated during training to achieve self-supervised learning.

[0041] Semi-manually labeled datasets: A batch of open-source or private object detection image data is collected. First, target images are captured from the target detection label files to form a target image set. An arbitrary image is selected from the open-source image dataset and then preprocessed, including brightness and contrast changes and noise addition. A randomly selected target screenshot is then pasted onto the preprocessed background image. The target screenshots are then uniformly positioned on the same background image to form an image sequence, simulating the motion of a real target in a dynamic video.

[0042] The target size in the same image sequence can vary, with the change not exceeding 10% of the previous geometric figure size. Adjacent background images in the same sequence are translated and rotated to simulate camera motion. The same sequence can only have one continuous translation or rotation or a combination of translation and rotation preprocessing. The change in translation / rotation between the previous and next frames does not exceed 5% of the image size. Finally, the target position (target circumscribed rectangular box) of each frame is automatically generated based on the position of the target map, and whether the target moves between adjacent images and the target displacement (c, Δx, Δy) are recorded.

[0043] Furthermore, semi-manually labeled sets can be generated during training to achieve self-supervised learning.

[0044] Fully manually annotated set: Continuous videos shot by fixed cameras and motion cameras in different scenes are collected, and moving targets are identified in the collected videos through manual annotation methods, and calibration tools are used for calibration.

[0045] Step B: Establishing a neural network parameter model, which includes an image input device, a feature extractor, and a structure output device connected in sequence.

[0046] The image input device is a first input structure, which includes two first downsampling convolutional layers arranged in parallel. They are structurally identical and consist of the same convolutional neural network layer, such as ResNet18. Two input images are respectively input into the two downsampling convolutional layers arranged in parallel to obtain a first feature map and a second feature map. The input images are adjacent image frames of a continuous video stream or images separated by N frames, where 0≤N≤50. The two feature maps are spliced ​​to obtain a first spliced ​​feature map. The input image resolution of the convolutional layer is W*H, and the output image resolution of the convolutional layer is W / 2*H / 2.

[0047] The feature extractor consists of multiple layers of sequentially connected downsampling convolutional layers and corresponding upsampling deconvolutional layers, each of which is connected by a corresponding channel concatenation layer. Furthermore, the feature extractor can also be modified to use other convolutional neural network structures, such as those based on mature convolutional neural network structures such as DarkNet53, VGG, ResNet, and GoogleNet.

[0048] The input of the feature extractor is the output feature map of the image input device. The dimension of the feature map is C*W / 2*H / 2, where C is the number of feature map channels, W is the width of the input image, and H is the height of the input image.

[0049] Furthermore, it includes five layers of downsampling convolutional layers and corresponding five layers of upsampling deconvolutional layers connected in sequence. The dimension of the input feature map is C*W / 2*H / 2. After one convolution, batch normalization, and activation function, the activation function is ReLu operation, and then the first layer of downsampling convolutional layer is downsampled to obtain a feature map with a dimension of C*W / 4*H / 4. The downsampling process can be a convolution with a step size of 2 or a maximum pooling with a window size of 2*2.

[0050] After repeated convolution, batch normalization, and activation function operations, a feature map with a dimension of 2C*W / 4*H / 4 is obtained. After downsampling through the second downsampling convolution layer, a feature map with a dimension of 2C*W / 8*H / 8 is obtained.

[0051] After convolution, batch normalization, and activation function operations, a feature map with a dimension of 4C*W / 8*H / 8 is obtained. After downsampling through the third downsampling convolution layer, a feature map with a dimension of 4C*W / 16*H / 16 is obtained.

[0052] After convolution, batch normalization, and activation function operations, a feature map with a dimension of 8C*W / 16*H / 16 is obtained. After downsampling through the fourth downsampling convolution layer, a feature map with a dimension of 8C*W / 32*H / 32 is obtained.

[0053] After convolution, batch normalization, and activation function operations, a feature map with a dimension of 16C*W / 32*H / 32 is obtained. After downsampling processing in the fifth downsampling convolution layer, a feature map with a dimension of 16C*W / 64*H / 64 is obtained. After one convolution, batch normalization, and activation function, the activation function is ReLu operation, and a feature map with a dimension of 32C*W / 64*H / 64 is obtained.

[0054] The fifth upsampling convolutional layer performs an upsampling operation, resulting in a 16C*W / 64*H / 64 feature map with fewer channels and larger width and height. The 2x upsampling operation can be either feature map interpolation or deconvolution. The upsampled feature map is concatenated with the corresponding downsampled feature map, and then subjected to convolution, batch normalization, and an activation function to produce a 16C*W / 64*H / 64 feature map output.

[0055] After the upsampling operation is performed in the fourth upsampling convolution layer, the upsampled feature map is spliced ​​with the corresponding feature map in the fourth downsampling layer, and then the 8C*W / 16*H / 16 feature map output is obtained through convolution, batch normalization, and activation function.

[0056] After the upsampling operation is performed in the third upsampling convolution layer, the upsampled feature map is spliced ​​with the corresponding feature map in the third downsampling layer, and then the 4C*W / 8*H / 8 feature map output is obtained through convolution, batch normalization, and activation function.

[0057] After the upsampling operation is performed in the second upsampling convolution layer, the upsampled feature map is spliced ​​with the corresponding feature map in the second downsampling layer, and then the 2C*W / 4*H / 4 feature map output is obtained through convolution, batch normalization, and activation function.

[0058] After the upsampling operation is performed in the first upsampling convolution layer, the upsampled feature map is spliced ​​with the corresponding feature map in the first downsampling layer, and then the C*W / 2*H / 2 feature map output is obtained through convolution, batch normalization, and activation function.

[0059] Furthermore, the number of convolution, batch normalization, and activation operations after each upsampling or downsampling to obtain a new feature map can be appropriately increased or decreased, and the number of downsampling layers and corresponding upsampling layers can also be increased or decreased.

[0060] The input of the structure output device is the output feature map of the feature extractor. The dimension of the feature map is C*W / 2*H / 2. The output feature map of the feature extractor is convolved and then convolved through three parallel convolutional layers to obtain the first output, second output, and third output respectively. The dimension of the first output is 12*W / 8*H / 8, the dimension of the second output is 12*W / 8*H / 8, and the dimension of the third output is 3*W / 8*H / 8. The first output, second output, and third output are used to represent the displacement between the target position in the first input frame image, the target position in the second input frame image, and the corresponding position target, respectively.

[0061] The first and second output target position definitions are the same as those of YOLOv3. The number of channels of each grid in the output feature map can be regarded as a 12-dimensional vector. The target position can be obtained by calculating every 4 elements of the vector with the preset anchor box. Therefore, each output feature map grid corresponds to 3 preset anchor boxes. The preset anchor boxes are the same as those defined in YOLOv3 and are pre-calculated based on the dataset. The third output feature map grid is a vector of dimension 3, defined as Where 0≤p≤1.0, p represents the probability of target movement, Indicates the offset of the moving target on the horizontal axis, Indicates the offset of the moving target on the vertical coordinate.

[0062] Step C: Train the neural network structure based on the acquired dataset and calculate the loss function until the loss function converges to a preset value to obtain a trained neural network parameter model. The neural network parameter model is trained using the automatically generated set, the semi-manually annotated set, and the fully manually annotated set, with corresponding parameter learning rates. Specifically, the automatically generated set randomly generates geometric shapes to simulate moving targets, the semi-manually annotated set randomly selects screenshots as moving targets, and the fully manually annotated set calibrates the moving targets.

[0063] Specifically include:

[0064] Model training is mainly divided into three stages: the first pre-training is performed using the automatically generated set, the second pre-training is performed using the semi-manually labeled set, and finally the correction training is performed using the fully manually labeled set to obtain the final deep convolutional neural network algorithm parameter model.

[0065] Step C1: Select image sequence data separated by n from the automatically generated set, denoted as Image 1 and Image 2, and uniformly scale the images to W*H pixel size as the input image, where n satisfies 0≤n≤50, W and H are m times 32, and m satisfies 15≤m≤20. Specifically, W and H are both set to 640, and n is set to 15.

[0066] Step C2: Perform forward reasoning calculations through the image input device, feature extractor, and structure output device to obtain the first output, second output, and third output, and calculate the loss function value L;

[0067] Furthermore, the image adjacent interval n is 15, that is, the selected input images are not adjacent but have intervals in between, and the target displacement parameter is calculated by accumulating the parameters corresponding to the images with intervals in between.

[0068] Step C3: Based on the obtained loss function value L, the parameter value is updated using the gradient to determine whether the training target is achieved. If so, the training is stopped. If not, check whether the preset number of training times is reached. If so, the training is stopped to obtain the first training model. After obtaining the first training model, the network parameters in the feature extractor are frozen and remain unchanged. Otherwise, steps C1 to C2 are repeated.

[0069] Step C4: Select image sequence data separated by n from the semi-manual annotation set, denoted as image 1 and image 2, and uniformly scale the images to W*H pixel size as input images, where n satisfies 0≤n≤50, W and H are m times 32, and m satisfies 15≤m≤20. Specifically, W and H are both set to 640, and n is set to 15.

[0070] Step C5: Set the parameter learning rate to 0.1 times the original value, perform parameter training based on the loss function value L to obtain the second training model, freeze the network parameters in the feature extractor and the network parameters of the image input device, so that the parameters are not updated during the parameter gradient descent training process.

[0071] Step C6: Select image sequence data with an interval of 15 from the fully manually annotated set, record them as image 1 and image 2, and perform the same image preprocessing as step C1.

[0072] Step C7: Set the parameter learning rate to 0.05 times of the original value, and perform parameter training based on the loss function value L to obtain the third training model, that is, to obtain the trained neural network parameter model.

[0073] Among them, the loss function, denoted as L, is calculated as

[0074] L=λ1L mov +λ2L distance +λ3L loc

[0075] L mov = -[c·log(p)+(1-c)log(1-p)]

[0076]

[0077]

[0078] Among them, L mov Indicates whether the position target has a moving loss value, L distance Indicates the displacement loss value of the target center, L loc Represents the loss value of the predicted moving target position, λ1 represents the weight of the loss value of whether there is a moving target, λ2 represents the weight of the displacement loss value of the target center, and λ3 represents the weight of the loss value of the predicted moving target position. The weights satisfy 5<λ1<10, 3<λ2<7, 0.5<λ3<1, where λ1=7, λ2=5, and λ3=0.7.

[0079] c represents the true value of whether the target moves in the label file, p represents the predicted value of whether the target moves in the label file, Δx represents the true value of the offset of the center of the moving target on the horizontal axis, and Δy represents the true value of the offset of the center of the moving target on the vertical axis. Indicates the predicted value of the offset of the moving target center on the horizontal coordinate, represents the predicted value of the offset of the center of the moving target on the vertical coordinate, x represents the true value of the horizontal coordinate of the center of the moving target stored in the label file, y represents the true value of the vertical coordinate of the center of the moving target stored in the label file, w represents the true value of the width of the moving target stored in the label file, h represents the true value of the height of the moving target stored in the label file, represents the predicted value of the central horizontal coordinate of the moving target, represents the predicted value of the central vertical coordinate of the moving target, represents the predicted value of the moving target width, They represent the predicted values ​​of the moving target height.

[0080] In step D, when using the trained neural network parameter model to detect moving targets on the test set, the image input in the trained neural network parameter model is a first input structure, the first input structure includes two parallel convolutional layers, such as ResNet18, the input image resolution is 640*640, and the output first feature map / second feature map resolution is 320*320. Step D specifically includes:

[0081] Step D1: Obtain video stream data captured by a camera, and decode the video stream data to obtain adjacent image frames of a continuous video stream or images separated by N frames, where 0≤N≤50, and obtain two three-channel images as input. Further, N=10.

[0082] Step D2: Input the two three-channel images into the neural network parameter model for moving target detection to obtain the moving target position.

[0083] Furthermore, the frame rate of the input video stream and the target movement rate are observed. If the frame rate of the input video stream is high and the target movement is slow, the number of intervals of the input image sequence is increased to 30 to obtain better results.

[0084] Verification experiment 1: According to the following experiment, a comparative test experiment was conducted on the mobile target detection method proposed by the present invention and the existing technology method for mobile targets. Experimental objects: 15 test videos, each test video is 2 minutes long, and the video resolution includes two different pixels: 1280*720 and 1920*1080. The mobile target detection algorithm of the present invention and the traditional mobile target detection method are used to obtain the mobile target detection results respectively, and the corresponding false alarm rate and detection rate are manually counted. The following Table 1 shows the test results of the mobile target detection method:

[0085] Table 1 Test results using different detection methods

[0086]

[0087] Analysis of experimental results:

[0088] 1. Compared with the existing technology methods, the method of the present invention has a higher detection rate and a lower false alarm rate, which fully demonstrates the effectiveness of the method of the present invention.

[0089] 2. When the feature extraction structure of the present invention is replaced with the backbone network daeknet53 of YOLOv4, although the false alarm rate and detection rate are slightly reduced in terms of indicators and the performance is slightly reduced, compared with the detection results of the traditional method with a false alarm rate of up to 29.3%, the method of the present invention still has extremely strong adaptability in the application scenario.

[0090] Verification experiment 2:

[0091] According to the following experiment, a comparative test experiment was conducted on the moving target detection method proposed by the present invention and the existing technology method for moving target detection under camera motion. Experimental objects: 13 test videos, each test video is between 30 seconds and 1 minute long, and the video resolution is 1920*1080 pixels. The moving target detection algorithm of the present invention and the traditional moving target detection method are used to obtain the moving target detection results for camera motion, and the corresponding false alarm rate and detection rate are manually counted. The following Table 2 shows the test results of the moving detection algorithm:

[0092] Table 2 Test results using different detection methods

[0093]

[0094] Analysis of experimental results:

[0095] 1. Compared with the existing methods, the method of the present invention has a higher detection rate and a lower false alarm rate in the case of camera motion, which fully demonstrates the effectiveness of the method of the present invention.

[0096] 2. When the feature extraction structure of the present invention is replaced with the backbone network daeknet53 of YOLOv4, in the case of camera motion, although the false alarm rate and detection rate show a slight decrease in performance, it still demonstrates adaptability to camera motion scenes and is still effective for detecting moving targets in camera motion scenes. In the case of camera motion, the false alarm rate of the existing method is as high as 81.1%, indicating that the traditional method mistakenly interprets background changes as target motion, resulting in a large number of false alarms, and is completely unsuitable for scenes with camera motion.

[0097] When used for intelligent moving target detection in dynamic backgrounds, the method of the present invention can reduce a large number of false alarms and provide more accurate results. For moving target detection in situations where the camera is moving, traditional methods cannot distinguish between background and foreground, resulting in a large number of false detections. In static backgrounds, traditional intelligent moving target detection methods still have a small number of false alarms, while the method of the present invention has fewer false alarms, fully demonstrating the effectiveness and robustness of the method of the present invention.

[0098] Example 2:

[0099] The steps of the second embodiment are basically the same as those of the first embodiment, except that:

[0100] In step B, the image input device in the established neural network parameter model is a second input structure, and the second input structure includes a second downsampling convolution layer. The input image resolution of the second input structure is W*H, and the output image resolution of the second input structure is W / 2*H / 2. The two input images are preprocessed separately to obtain a first single-channel grayscale image and a second single-channel grayscale image, and the two preprocessed input images are spliced ​​and combined into a three-channel image, wherein channel 1 is the first single-channel grayscale image, and channels 2 and 3 are the second single-channel grayscale images. And input into the second downsampling convolution layer to obtain a third feature map. Furthermore, the input image resolution is 640*640, and the output resolution of the third feature map is 320*320. The third feature map has the same number of channels as the first feature map and the second feature map.

[0101] In step D, when using the trained neural network parameter model to detect moving targets on the test set, the following steps are specifically included:

[0102] Step D1 ′: obtaining video stream data captured by a camera, and decoding the video stream data to obtain a continuous video frame sequence.

[0103] Step D2': select two frames of images separated by 10 frames from the video frame sequence, and scale the images to the same size as the deep convolutional network input, 640*640, and then preprocess the images to obtain a first single-channel grayscale image and a second single-channel grayscale image.

[0104] Step D3': Combine the two single-channel grayscale images into a three-channel image, where channel 1 is the first single-channel grayscale image, and channels 2 and 3 are both second single-channel grayscale images, and input the three-channel image into the neural network parameter model for moving target detection to obtain the moving target position.

[0105] Furthermore, the frame rate of the input video stream and the target movement rate are observed. If the frame rate of the input video stream is high and the target movement is slow, the number of intervals of the input image sequence is increased to 30 to obtain better results.

[0106] Example 3:

[0107] A computer-readable storage medium storing computer instructions for causing a computer to execute a moving target detection method based on semi-supervised learning as proposed in Example 1. The artificial intelligence acceleration hardware may be Huawei's Atlas 200 neural network-specific inference hardware. The network model provided in Example 1 is converted into a .om format file using a model conversion toolchain provided by Huawei, so that the model parameters can be correctly loaded into the computer-readable storage medium for inference calculation.

[0108] Example 4:

[0109] An electronic device, which can be a single server or an embedded computing platform, includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform a moving target detection method based on semi-supervised learning as proposed in Example 1.

[0110] The memory is a machine-readable storage medium. The electronic device also includes a deep learning parallel computing acceleration chip and a network interface. The deep learning parallel computing acceleration chip, the machine-readable storage medium, the network interface and the processor are connected through a PCIe bus system. The deep learning parallel computing acceleration chip is used to accelerate the forward reasoning calculation of the deep learning model. The machine-readable storage medium is used to store programs, instructions or codes. The processor can be used to control the sending and receiving actions of the network interface, so that data can be sent and received through the network. The deep learning parallel computing acceleration chip realizes parallel processing of the front-line reasoning calculation of the deep learning network, thereby speeding up the calculation speed.

[0111] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection of the present invention.

Claims

1. A moving target detection method based on semi-supervised learning, characterized in that: Including steps: A. Obtain a dataset, which includes an automatically generated set, a semi-manually annotated set, a fully manually annotated set, and a test set. The semi-manually annotated set consists of randomly selecting a target screenshot from the target image set and pasting it onto a preprocessed background image. The position of the target screenshot is uniformly shifted on the same background image to form an image sequence. Adjacent background images in the same sequence are translated and rotated. B. Establish a neural network parameter model, which includes an image input device, a feature extractor, and a structure output device connected in sequence; the feature extractor includes multiple layers of downsampling convolution layers and corresponding multiple layers of upsampling deconvolution layers connected in sequence, and corresponding channel splicing layers are connected between the downsampling convolution layers and the corresponding upsampling deconvolution layers; the structure output device includes three parallel convolution layers, corresponding to the three convolution results respectively, the first output represents the target position in the first frame of the input image, the second output represents the target position in the second frame of the input image, and the third output represents the displacement between the corresponding position targets; C. Training the neural network parameter model based on the automatically generated set, semi-manually annotated set, and fully manually annotated set in the acquired data set, and calculating the loss function until the loss function converges to a preset value to obtain a trained neural network parameter model; D, use the trained neural network parameter model to detect moving targets on the test set; Among them, the model training of the neural network parameter model is divided into three stages: the first pre-training is performed using the automatically generated set, the second pre-training is performed using the semi-manually labeled set, and finally the correction training is performed using the fully manually labeled set to obtain the final neural network parameter model.

2. The method for detecting moving objects based on semi-supervised learning according to claim 1, wherein: The image input device includes two first convolution layers and a feature splicing layer arranged in parallel. The two first convolution layers are respectively used to input the first image and the second image to obtain two feature maps. The feature splicing layer is used to splice the two feature maps to obtain a first spliced ​​feature map.

3. The method for detecting moving objects based on semi-supervised learning according to claim 1, wherein: The image input device includes a feature stitching layer and a second convolution layer connected in sequence. The feature stitching layer is used to stitch two preprocessed input images. The preprocessing is to process the input image into a single-channel grayscale image to obtain a stitching image. The second convolution layer is used to convolve the obtained stitching image to obtain a second stitching feature map.

4. A method for detecting moving objects based on semi-supervised learning according to any one of claims 1 to 3, characterized in that: In step C, two sequential images separated by a preset number of frames are selected from the acquired data set and scaled to obtain two images with the same pixel size for training the neural network parameter model.

5. The method for detecting moving objects based on semi-supervised learning according to claim 4, wherein: In step C, when training with the automatically generated set, semi-manually labeled set, and fully manually labeled set in turn, the corresponding parameter learning rate is set. The automatically generated set randomly generates geometric figures to simulate moving targets, the semi-manually labeled set randomly selects screenshots as moving targets, and the fully manually labeled set manually calibrates the moving targets.

6. A method for detecting moving objects based on semi-supervised learning according to any one of claims 1 to 3, characterized in that: In step C, the loss function, denoted as L, is calculated as follows: in, Indicates whether the moving target has a loss value. Indicates the displacement loss value of the moving target center, Represents the loss value of the moving target position prediction, The weight of the loss value indicating whether the moving target is moving, Represents the weight of the displacement loss value of the moving target center, Represents the weight of the loss value of the moving target position prediction, Indicates the true value of whether the moving target is moving, Indicates the predicted value of whether the moving target is moving, Indicates the true value of the offset of the moving target center on the horizontal axis, Indicates the true value of the offset of the moving target center on the vertical coordinate, Indicates the predicted value of the offset of the moving target center on the horizontal axis, Indicates the predicted value of the offset of the moving target center on the vertical coordinate, Represents the true value of the central horizontal coordinate of the moving target, Represents the true value of the central vertical coordinate of the moving target, represents the true value of the moving target width, Indicates the true value of the moving target height, Represents the predicted value of the central horizontal coordinate of the moving target, Represents the predicted value of the central vertical coordinate of the moving target, Indicates the predicted value of the moving target width, Indicates a high predicted value for a moving target.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the moving target detection method based on semi-supervised learning as described in any one of claims 1 to 6.

8. An electronic device, characterized in that: It includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute a motion target detection method based on semi-supervised learning as described in any one of claims 1 to 6.

Images