Floating object recognition method based on improved SSD model and storage medium

Abstract

The application provides a floating object identification method based on an improved SSD model, which inputs image data into an optimized SSD model to identify target floating objects in the image, wherein the training process of the optimized SSD model comprises the following steps: collecting original water surface image data, and obtaining an original sample set after disorder processing; performing image enhancement processing on the original samples in the original sample set to obtain an expanded sample set; manually labeling floating object information of the expanded samples in the expanded sample set; and using sample data in the expanded sample set to train a pre-trained optimized SSD model, wherein the optimized SSD model comprises multiple small target detection layers. Through the enhancement processing of the water surface image data and the small target optimized SSD model, the method can better eliminate the regularity differences in the training image and avoid affecting the model accuracy. The lightweight SSD detection method can meet the real-time requirement of detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a floating object recognition method and storage medium based on an improved SSD model. Background Technology

[0002] Currently, cameras have been installed around some waterways to monitor the water surface, but personnel are still needed to monitor the footage. This is not only time-consuming and labor-intensive, but also cannot guarantee accurate and real-time responses to floating objects. In addition, personnel patrols are inevitably affected by the long distances of the waterways, the long patrol times, and blind spots, resulting in floating objects not being detected in a timely manner, or even being missed.

[0003] To improve the efficiency of river problem detection and reduce the workload of manual river inspection, it is necessary to develop an automatic method for identifying floating objects on the water surface. While the SSD target detection algorithm based on neural networks has a certain advantage in detection speed, its detection accuracy is insufficient for direct application in the field of floating object identification. The reason is as follows:

[0004] 1. Individual floating objects on the water surface are relatively small in size, so each floating object occupies a small pixel area in the field of view. During the detection algorithm's recognition process, the feature information of the floating objects is easily lost. Since the lateral area of ​​the floating objects exposed on the water surface is often even smaller, this problem is more obvious in side-shot image data.

[0005] 2. Ripples on the water surface dynamically reflect sunlight, forming a large number of randomly and irregularly arranged light spots within the field of view. These light spots are basically on the same order of magnitude as the size of the floating object, which will interfere with the performance of the detection algorithm, leading to false detections and affecting the detection accuracy.

[0006] 3. Due to different detection times or different weather conditions, the lighting conditions of the images are different, resulting in significant differences in brightness, making it difficult to guarantee good performance of the detection algorithm under different lighting conditions. Summary of the Invention

[0007] To address the technical problem that existing methods for detecting floating objects on the water surface have low accuracy and cannot meet application requirements, this invention provides a floating object identification method and storage medium based on an improved SSD model.

[0008] The technical solution of this invention provides a floating object identification method based on an improved SSD model.

[0009] The floating object recognition method based on the improved SSD model of this invention, through enhanced processing of water surface image data and the adoption of an improved SSD model tailored to the characteristics of floating objects, can effectively eliminate regular differences in training images, avoiding the introduction of excessive noise into the model and affecting its accuracy. Furthermore, while reducing training parameters, it can better identify the features of floating objects, improving recognition accuracy. Due to the use of a lightweight SSD detection method, it also meets the real-time requirements for detection efficiency. Attached Figure Description

[0010] Figure 1 This is an architecture diagram of the improved SSD model of the present invention;

[0011] Figure 2 This is a structural diagram of the small target detection layer of the present invention;

[0012] Figure 3 This is a schematic diagram illustrating the principle of dilated convolution in this invention;

[0013] Figure 4 This is the convergence curve of the loss function of this invention. Detailed Implementation

[0014] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0015] This invention primarily aims to address the issue of insufficient accuracy in existing water surface floating object recognition algorithms. To achieve this goal, this application focuses on the synergistic optimization of image enhancement processing and the SSD detection algorithm. The floating object recognition method based on the improved SSD model of this invention uses a floating object recognition model trained on an enhanced training set with an SSD detection algorithm optimized for the water surface floating object recognition problem. This model then detects floating objects in the image-processed water surface image, achieving accurate recognition of floating objects.

[0016] Specifically, the floating object recognition method based on the improved SSD model of this invention first requires obtaining an image recognition model applicable to a production environment. The steps for generating the image recognition model include:

[0017] SS1. The original water surface image data is obtained by shuffling the collected raw water surface image data. The raw water surface image data mainly comes from river or lake water surface image data obtained through drones, monitoring equipment, etc. To ensure the representativeness of the image data, the raw water surface image data should include water surface image scenes under different lighting conditions, weather conditions, and water quality conditions as much as possible. Here, the raw water surface image data refers to image data, which can be a single frame image extracted from the collected video data or independently collected image data. To prevent multiple frames extracted from the same video data from reducing the convergence speed of the subsequent training process or even leading to local optima, the raw water surface image data needs to be randomly shuffled. In the embodiments of this invention, the number of raw samples is not less than 20,000.

[0018] SS2. Image enhancement processing is performed on the original samples obtained in step SS1 to obtain an expanded sample set. Image processing mainly aims to eliminate regular differences in the original image data, avoiding their adverse effects on the training process and reducing the accuracy of the obtained image recognition model; alternatively, new samples can be generated from the original samples through transformation, thereby enriching the number of expanded samples. Optionally, as described in the background of this invention, image data under different lighting conditions exhibit significant brightness differences. A feasible enhancement scheme is to average the brightness of the original samples. This reduces the impact while compensating for the brightness of image data under poor lighting conditions, reducing the interference of brightness differences on the model. The enhancement process also includes generating new expanded samples based on the original samples through geometric transformation, color transformation, and the SMOTE (Synthetic Minority Over-Sampling Technique) method. These expanded samples, together with the original samples, constitute a complete expanded sample set, thus enriching the sample set based on the original sample set. SMOTE can synthesize new samples by interpolating small sample classes based on the original samples. Therefore, after adding new samples, it can better solve the problem of imbalance of the original samples, thereby improving the classification performance of the trained model and increasing the model accuracy.

[0019] In the process of image enhancement, the geometric transformation of the original samples to generate new samples includes the transformation of the original samples...

[0020]

[0021] Table 1 Parameter settings for each feature layer

[0022] A new expanded sample is obtained by randomly applying a horizontal flip operation with a first probability; a new expanded sample is obtained by randomly scaling the original sample horizontally and / or vertically with a second probability. In this embodiment, the scaling factor in the random horizontal and / or vertical scaling step ranges from 0.8 to 1.4. The values ​​of the first probability and the second probability both range from 0.3 to 0.7, preferably 0.6.

[0023] SS3. The extended samples in the extended sample set are manually labeled, i.e., the floating objects are manually identified and bounded with a bounding box of the same size as the floating objects. Then, the images of the manually labeled extended samples are adjusted to the same resolution. Depending on the actual conditions, the resolution can be determined by those skilled in the art based on a balance between accuracy and efficiency. In this application, the input image resolution is 300*300. Optionally, the extended samples can be Gaussian filtered before adjusting the image resolution to eliminate random noise in the data. Then, the extended samples are divided into a training set, a validation set, and a test set according to a certain ratio. The training set and validation set are used to train the parameters based on the improved SSD model, and the test set is used to test the final recognition model and evaluate its performance. In this invention, the ratio of the number of samples in the training set, validation set, and test set is 7:2:1.

[0024] SS4. Model training process: The pre-trained improved SSD model is used as the source model. The parameters of the source model are trained using the training set in step SS3. The trained source model is then validated using the validation set. Based on the validation error, it is determined whether to adjust the model to continue training or stop training, thus obtaining the final recognition model.

[0025] Among them, the improved architecture of the SSD model is as follows: Figure 1 As shown in Table 1, the parameter settings for each feature layer are shown in Table 1 (where the offset value represents the horizontal and vertical proportion of the distance between the point in the candidate box and the top left corner of the mapping map). The improved SSD model replaces several low-level feature layers from the original convolutional feature layers with the small object detection layers L3, L4, and L7 described in this application.

[0026] The overall training process is consistent with the SSD model. In this model, the input image undergoes a series of operations, including convolution and pooling in the conv4_3 layer, resulting in a 38×38×512 feature map. Then, this feature map is used for classification and regression prediction with a default of 4 prior boxes per grid point, yielding 38×38×4=5776 prior boxes. After the feature map size is further compressed to 19×19 by pooling in the conv4_3 convolution, the Fc7 convolutional layer begins processing the output image. Next, all these feature maps are used for classification and regression prediction with a default of 6 prior boxes per grid point, yielding 19×19×6=2166 prior boxes. The above process is similar to the VGG16-based base network in the general SSD algorithm. The resulting feature images are then processed by the SODB module, which replaces the original SSD model, through multi-layer convolution. Each convolutional layer uses dilated convolutions with different dilation rates to generate feature maps with larger receptive fields while preserving the original image features, thereby optimizing the recognition rate of floating objects on the water surface and improving computational efficiency. The SODB_1 layer, which replaces the conv6-2 convolution in the original SSD model, further compresses the feature maps to a size of 10×10 for processing, generating feature maps with even larger receptive fields. These feature images are then subjected to classification and regression prediction with 6 prior boxes per grid point (default), resulting in 10×10×6=600 prior boxes. Similarly, after processing by the subsequent SODB_2 and SODB_3 layers, the feature images generated by SODB_3 are subjected to classification and regression prediction with 6 prior boxes per grid point (default), resulting in 150 prior boxes. The subsequent two convolutional layers are at the top of the model pyramid and are mainly used to obtain large-size and more abstract features of the image. They are not very useful for the floating object scene in this application. Therefore, the original model is kept unchanged. The output feature image size of conv8-2 layer is 3×3 and the output feature image size of conv9-2 layer is 1×1.

[0027] Specifically, the structure of the SODB layer is as follows: Figure 2 As shown, the input feature image is processed by dilated convolutions at different scales, the results are aggregated, and then an activation function ReLU is applied for delinearization. This nonlinear mapping prevents gradient vanishing and reduces overfitting. The feature parameters processed by the SODB layer are then fed into the Detector, and the multi-layered detector increases the object detection accuracy.

[0028] For dilated convolutions, a hyperparameter called "dilation rate" is introduced. Figure 3From left to right, the images show the original image, a dilated convolution with a dilation rate of 1, and a dilated convolution with a dilation rate of 3. The spacing between each value in the convolution kernel's calculations is defined by this parameter. Due to its unique structure, dilated convolution increases the receptive field without changing the feature map size. Furthermore, multi-scale information can be obtained by using various different dilated convolution structures. The receptive field of the 3x3 convolution kernel in the original SSD detection model is too large, resulting in the loss of original information. Repeated convolutions can destroy details in the original image, causing the loss of many features and affecting detection accuracy, particularly for small targets. The advantage of dilated convolution is that each convolution output can include a large range of information. This method of increasing the receptive field is particularly beneficial for improving the recognition of small floating objects on the water surface, significantly increasing the detection rate of small targets among these objects.

[0029] The SODB layer of this invention combines three types of 3x3 dilated convolutions with rates of 1, 3, and 5 to increase the receptive field and improve the recognition rate of local features. Specifically, it extracts original image information by aggregating dilated convolutions with dilation rates of 1, 3, and 5 respectively, and fuses multiple scales to achieve better recognition results. The 1x1 convolutional layers at the beginning and end are mainly used to change the dimension of the feature map. The combination of 3x1 and 1x1 layers avoids the use of 3x3 convolution kernels, reduces the number of parameters, increases the model's computation speed, and increases the model's feature extraction speed.

[0030] The experimental platform information for the improved SSD model of this invention is shown in Table 2.

[0031] Model parameter settings are shown in Table 3

[0032] Figure 4 Table 4 shows the loss function reduction of the original SSD model and the improved SSD model of this invention, respectively. Table 4 also shows the recognition rates of the two algorithms on the same test set.

[0033] For deep learning-based image recognition, the loss function value in the experiment represents the distance (metric) between the model output and the true result. With continuous iterative training, the model's loss will continuously decrease, updating the network parameters. The lower the loss value, the better the model's performance.

[0034]

[0035] Operating platform information

[0036]

[0037] Model parameter settings

[0038] from Figure 4 As can be seen, the original SSD model had a training loss convergence value of 4.4893 and a validation loss convergence value of 5.5838, while the improved SSD model had a training loss convergence value of 3.4431 and a validation loss convergence value of 4.4293. The original SSD model converged after 85 epochs, while the improved model converged after 65 epochs, significantly improving the loss convergence speed and greatly reducing the loss value. A smaller validation loss value indicates better model robustness and better performance in recognizing floating objects.

[0039] As can be seen from the results in Table 3, the improved SSD algorithm, which addresses the characteristics of water surface floating object images with their chaotic size and numerous small targets, improved the recognition rate from 85.5% to 93.1% based on the test set, significantly enhancing the recognition rate of water surface floating objects.

[0040] S1. Acquire water surface image data of the monitored water area from an image acquisition device, which may be a monitoring camera deployed at a predetermined location or aerial data from an aerial drone.

[0041] S2. Preprocess the image data to form an input image that conforms to the improved SSD model. This mainly includes resetting the image resolution and adjusting the image brightness, adjusting the image brightness and image resolution to the set values ​​used when training the model.

[0042] S3. Input the image data into the improved SSD model to obtain the center coordinates and target bounding box size data of the target floating object in the image, thereby determining the floating object.

[0043] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying floating objects based on an improved SSD model, characterized in that, comprising the following steps: obtaining image data of a monitored water area collected by an image collection device; preprocessing the image data to form input image data conforming to the improved SSD model; inputting the input image data into the trained improved SSD model to identify the target floating object in the image data, wherein the improved SSD model comprises at least one small target detection layer for targeted detection of floating objects on the basis of the SSD model, the small target detection layer is three layers, which are sequentially arranged after the pooling layer of the VGG base network of the improved SSD model, and two convolution feature layers are further arranged after the small target detection layer; the small target detection layer is aggregated by one dilated convolution with an expansion rate of 1, two dilated convolutions with an expansion rate of 3, and two dilated convolutions with an expansion rate of 5; the small target detection layer further comprises a 3*1 convolution layer and a 1*1 convolution layer arranged in sequence; the training process of the improved SSD model is as follows: obtaining the original sample set by randomly processing the image data containing floating objects; performing image enhancement processing on the image data in the original sample set to obtain an expanded sample set; manually labeling the floating object information in the image data in the expanded sample set to obtain a labeled training sample set; training the improved SSD model using the training sample set. 2.The method for identifying floating objects based on the improved SSD model according to claim 1, characterized in that, the step of performing image enhancement processing on the original sample in the original sample set to obtain an expanded sample set comprises performing average processing on the image brightness of the original sample. 3.The method for identifying floating objects based on the improved SSD model according to claim 1, characterized in that, the step of performing image enhancement processing on the original sample in the original sample set to obtain an expanded sample set further comprises: applying a horizontal flip operation to the original sample with a first probability to obtain a new expanded sample; and applying random horizontal and / or vertical scaling to the original sample with a second probability to obtain a new expanded sample, wherein the scaling range of the scaling factor in the step of applying random horizontal and / or vertical scaling is 0.8-1.4, and the value range of the first probability and the second probability is 0.3-0.

7. 4.A computer storage medium, characterized in that, the computer storage medium stores a computer program, and the computer program is executed by a processor to make the processor execute the improved SSD model or the method for identifying floating objects based on the improved SSD model according to any one of claims 1-3. ​

Images