A target classification method for complex scenes

Abstract

The present invention discloses a target classification method for complex scenes, comprising: collecting images to be trained and images to be classified; preprocessing the collected images; using the preprocessed images to be trained to create a data set; training a target classification model; using the target classification model to process the preprocessed images to be classified; and displaying the target classification results. The present invention crawls image information containing complex scenes from multiple websites through a crawling module, and creates a data set after screening the crawled images, thereby ensuring the quantity and quality of training data. At the same time, by training the target classification model and using the target classification model to process the preprocessed images to be classified, the target classification results can be efficiently obtained.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target classification method for complex scenes. Background Art

[0002] Nowadays, information carriers are diverse. Among all forms of carriers, images are the most common. With the help of computers and classification algorithms, people can quickly identify targets in images, greatly reducing manpower and material costs. However, when the acquired image is blurry or in places with poor lighting or objects are obscured, it is often impossible to accurately identify the target from the image data. Therefore, in complex scenarios, target classification is extremely necessary.

[0003] Therefore, providing a new technical solution to improve the above problems is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0004] In view of this, the present invention provides a target classification method for complex scenarios to solve the above technical problems.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A target classification method for complex scenes includes: collecting images to be trained and images to be classified; preprocessing the collected images; using the preprocessed images to be trained to create a data set; training a target classification model; using the target classification model to process the preprocessed images to be classified; and displaying the target classification results.

[0007] In the above scheme, the acquisition of images to be trained includes the following steps: crawling image information containing complex scenes from multiple websites through a crawling module; managing the crawling process through a crawler management module; storing the crawled image information through a storage module, the crawling module includes a downloading unit, a parsing unit, a memory unit and a database unit, the downloading unit is used to download the web page to be crawled, the parsing unit is connected to the downloading unit, the parsing unit is used to parse useful data from the web page downloaded by the downloading unit and save it into an object, the memory unit is connected to the parsing unit, the memory unit is used to persist the object parsed by the parsing unit, the database unit is connected to the memory unit, the database unit is used to store the data processed by the memory unit; the crawler management module includes a status display unit and a crawler control unit, the status display unit is used to display the specific status of the crawling process, the crawler control unit is used to control the start, pause and crawling interval of the crawler; the storage module includes a computer hard disk.

[0008] In the above scheme, the acquisition of images to be classified includes acquiring image information to be classified through a camera module and storing the acquired image information to be classified in a storage module. The camera module includes a CMOS sensor, an electric zoom lens, a zoom drive motor, an LED light group, a relay, a brightness sensor, an infrared sensor and a pan-tilt head. The CMOS sensor, the electric zoom lens, the zoom drive motor, the LED light group, the relay, the brightness sensor and the infrared sensor are all installed on the pan-tilt head. The CMOS sensor is used to acquire image information, the electric zoom lens is connected to the CMOS sensor, the zoom drive motor is used to drive the electric zoom lens to zoom, the relay is connected to the LED light group, the relay is used to drive the LED light group to illuminate, the brightness sensor is used to acquire ambient brightness information, and the infrared sensor is used to detect whether there is a moving target in the monitoring area.

[0009] In the above scheme, the preprocessing of the collected images includes preprocessing the collected images to be trained, and the preprocessing of the collected images to be trained includes the following steps: normalizing the collected images to be trained to process the images to be trained with the same specifications; setting a feature point threshold, obtaining the number of feature points of the images to be trained, and deleting the images to be trained whose number of feature points is greater than the feature point threshold; performing feature binarization processing on the images to be trained remaining after deletion through a fuzzy mean binarization algorithm to separate the target to be detected from the background; calculating the proportion of the target to be detected in the entire image, and deleting the images to be trained whose proportion of the target to be detected in the entire image is less than a preset ratio.

[0010] In the above scheme, the preprocessing of the collected images also includes preprocessing the collected images to be classified, and the preprocessing of the collected images to be classified includes the following steps: performing distortion correction processing on the images to be classified through a convolutional neural network algorithm; performing noise reduction processing on the images to be classified after the distortion correction processing through a median filtering algorithm; and performing image grayscale processing on the images to be classified after the noise reduction processing.

[0011] In the above scheme, the use of preprocessed images to be trained to produce a data set includes the following steps: performing a reduction operation on the preprocessed images to be trained using the resize function of the OpenCv library; dividing the images to be trained after the reduction operation into two parts according to a preset ratio, using one part as a training data set and the other part as a test data set; packaging the training data set into a training TFRecords picture set, and packaging the test data set into a test TFRecords picture set.

[0012] In the above scheme, the training target classification model includes the following steps: training the target detection network through the SSD algorithm and a data set made using the preprocessed images to be trained; training the target classification network through the convolutional neural network algorithm and a data set made using the preprocessed images to be trained.

[0013] In the above scheme, the use of the target classification model to process the preprocessed image to be classified includes inputting the preprocessed image to be classified into the target detection network for target detection, and the inputting the preprocessed image to be classified into the target detection network for target detection includes the following steps: extracting features from the preprocessed image to be classified through a feature extraction module to obtain a feature map; performing transposed convolution on the low layer of the feature extraction module, and calculating the resolution of the low-layer feature map obtained after the transposed convolution; repeating the transposed convolution process until the resolution of the low-layer feature map of the feature extraction module is consistent with the resolution of the high-layer feature map of the feature extraction module; fusing the high-layer feature map and the low-layer feature map through a fusion convolution module; inputting the feature map output by the fusion convolution module into a convolution predictor for prediction; and selecting the best prediction result through a maximum suppression algorithm. The feature extraction module includes multiple pairs of a pooling layer and at least one convolution layer.

[0014] In the above scheme, the use of the target classification model to process the pre-processed image to be classified also includes inputting the target obtained through the target detection network into the target classification network for target classification, and the inputting the target obtained through the target detection network into the target classification network for target classification includes the following steps: extracting features of the target obtained by the target detection network through an ordinary convolution module and a Gabor convolution module respectively; splicing the feature map output by the ordinary convolution module and the feature map output by the Gabor convolution module to obtain a new feature map; performing a dimensionality increase operation on the new feature map; adding the feature map after dimensionality increase and the target obtained by the target detection network through a summation operation; summing the results obtained by the summation operation through a global average pooling layer, and completing the classification task through a softmax classifier.

[0015] In the above-mentioned scheme, the display of the target classification results includes displaying the target classification results through a display module and sending the target classification results to a remote terminal for display. The display module includes an LCD touch screen, a physical button and a power indicator light. The physical button and the power indicator light are both connected to the LCD touch screen. The physical button is used to turn on and off the LCD touch screen, and the power indicator light is used to indicate the power connection status of the LCD touch screen.

[0016] To sum up, the beneficial effects of the present invention are: by using the crawling module to crawl image information containing complex scenes from multiple websites, and then screening the crawled images to create a data set, the quantity and quality of the training data are guaranteed; at the same time, by training the target classification model and using the target classification model to process the pre-processed images to be classified, the target classification results can be obtained efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0018] Figure 1 This is a step diagram of the target classification method for complex scenarios in the present invention.

[0019] Figure 2 This is a diagram of the steps for collecting images to be trained in the present invention.

[0020] Figure 3 This is a schematic diagram of the composition of the crawling module in the present invention.

[0021] Figure 4 This is a diagram of the steps for collecting images to be classified in the present invention.

[0022] Figure 5 Schematic diagram of the composition of the camera module in the present invention.

[0023] Figure 6 This is a diagram of the steps for preprocessing the collected images to be trained in the present invention.

[0024] Figure 7 This is a diagram of the steps for preprocessing the collected images to be classified in the present invention.

[0025] Figure 8 A diagram showing the steps for creating a dataset using pre-processed training images in the present invention.

[0026] Figure 9 This is a diagram of the steps for training the target classification model in the present invention.

[0027] Figure 10 Schematic diagram of target detection steps in the present invention.

[0028] Figure 11 A diagram showing the steps for classifying objects in the present invention.

[0029] Figure 12 It is a schematic diagram of the composition of the display module in the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] like Figure 1 As shown, the target classification method for complex scenes of the present invention includes the following steps:

[0032] Step S1: Collect images to be trained and images to be classified;

[0033] Step S2: pre-processing the collected image;

[0034] Step S3: Use the pre-processed images to be trained to create a data set;

[0035] Step S4: training the target classification model;

[0036] Step S5: using the target classification model to process the pre-processed image to be classified;

[0037] Step S6: Display the target classification results.

[0038] like Figure 2 As shown, the acquisition of the image to be trained includes the following steps:

[0039] Step S111: crawling image information containing complex scenes from multiple websites through a crawling module;

[0040] Step S112: managing the crawling process through the crawler management module;

[0041] Step S113: The crawled image information is stored through the storage module.

[0042] like Figure 3 As shown, the crawling module includes a downloading unit, a parsing unit, a memory unit and a database unit. The downloading unit is used to download the web page to be crawled. The parsing unit is connected to the downloading unit. The parsing unit is used to parse out useful data from the web page downloaded by the downloading unit and save it into an object. The memory unit is connected to the parsing unit. The memory unit is used to persist the object parsed by the parsing unit. The database unit is connected to the memory unit. The database unit is used to save the data processed by the memory unit.

[0043] Furthermore, the crawler management module includes a status display unit and a crawler control unit, the status display unit is used to display the specific status of the crawler process, and the crawler control unit is used to control the start, pause and crawling interval of the crawler; the storage module includes a computer hard disk.

[0044] like Figure 4 As shown, the acquisition of images to be classified includes the following steps:

[0045] Step S121: acquiring image information to be classified through a camera module;

[0046] Step S122: storing the acquired image information to be classified into a storage module.

[0047] like Figure 5 As shown, the camera module includes a CMOS sensor, an electric zoom lens, a zoom drive motor, an LED light group, a relay, a brightness sensor, an infrared sensor and a pan-tilt head. The CMOS sensor, the electric zoom lens, the zoom drive motor, the LED light group, the relay, the brightness sensor and the infrared sensor are all installed on the pan-tilt head. The CMOS sensor is used to collect image information, the electric zoom lens is connected to the CMOS sensor, the zoom drive motor is used to drive the electric zoom lens to zoom, the relay is connected to the LED light group, the relay is used to drive the LED light group for lighting, the brightness sensor is used to obtain ambient brightness information, and the infrared sensor is used to detect whether there is a moving target in the monitoring area.

[0048] like Figure 6 As shown, the preprocessing of the collected images includes preprocessing the collected images to be trained, and the preprocessing of the collected images to be trained includes the following steps:

[0049] Step S211: normalize the collected images to be trained to the same specifications;

[0050] Step S212: setting a feature point threshold, obtaining the number of feature points of the image to be trained, and deleting the image to be trained with a number of feature points greater than the feature point threshold;

[0051] Step S213: performing feature binarization processing on the remaining training image after deletion using a fuzzy mean binarization algorithm to separate the target to be detected from the background;

[0052] Step S214: Calculate the ratio of the target to be detected to the entire image, and delete the training images in which the ratio of the target to be detected to the entire image is less than a preset ratio value.

[0053] like Figure 7As shown, the preprocessing of the collected images also includes preprocessing the collected images to be classified, and the preprocessing of the collected images to be classified includes the following steps:

[0054] Step S221: performing distortion correction processing on the image to be classified using a convolutional neural network algorithm;

[0055] Step S222: performing noise reduction processing on the image to be classified after the distortion correction processing by using a median filtering algorithm;

[0056] Step S223: performing image grayscale processing on the image to be classified after the noise reduction processing.

[0057] like Figure 8 As shown, the method of using the pre-processed images to be trained to create a data set includes the following steps:

[0058] Step S31: performing a reduction operation on the pre-processed image to be trained through the OpenCv library resize function;

[0059] Step S32: dividing the image to be trained after the reduction operation into two parts according to a preset ratio, using one part as a training data set and the other part as a test data set;

[0060] Step S33: Package the training dataset into a training TFRecords picture set, and package the test dataset into a test TFRecords picture set.

[0061] In this embodiment, the reduction operation includes the following steps: reading the preprocessed image to be trained through the OpenCv library; setting the cropping vertex value of the image read by the OpenCv library; and cropping the image read by the OpenCv library according to the cropping vertex value through the resize function.

[0062] like Figure 9 As shown, the training target classification model includes the following steps:

[0063] Step S41: training the object detection network using the SSD algorithm and a data set made using the pre-processed training images;

[0064] Step S42: training a target classification network using a convolutional neural network algorithm and a data set made using the pre-processed training images.

[0065] In this embodiment, the target detection network includes a feature extraction module, a transposed convolution module, a fused convolution module and a convolution predictor, the transposed convolution module is connected to the feature extraction module, the fused convolution module is connected to the transposed convolution module, and the convolution predictor is connected to the fused convolution module.

[0066] In this embodiment, the target classification network includes a normal convolution module, a Gabor convolution module, a splicing module, a dimensionality increase operation module, a summation operation module, a global average pooling layer and a softmax classifier. The splicing module is connected to the normal convolution module and the Gabor convolution module, the dimensionality increase operation module is connected to the splicing module, the summation operation module is connected to the dimensionality increase operation module, the global average pooling layer is connected to the summation operation module, and the softmax classifier is connected to the global average pooling layer.

[0067] like Figure 10 As shown, the processing of the preprocessed image to be classified using the target classification model includes inputting the preprocessed image to be classified into a target detection network for target detection, and the inputting the preprocessed image to be classified into the target detection network for target detection includes the following steps:

[0068] Step S511: extracting features from the pre-processed image to be classified by a feature extraction module to obtain a feature map;

[0069] Step S512: performing transposed convolution on the lower layer of the feature extraction module, and calculating the resolution of the lower layer feature map obtained by the transposed convolution;

[0070] Step S513: Repeat the transposed convolution process until the resolution of the low-level feature map of the feature extraction module is consistent with the resolution of the high-level feature map of the feature extraction module;

[0071] Step S514: Fusing the high-level feature map and the low-level feature map through the fusion convolution module; inputting the feature map output by the fusion convolution module into the convolution predictor for prediction;

[0072] Step S515: Select the best prediction result through the maximum value suppression algorithm.

[0073] Furthermore, the feature extraction module includes multiple pairs of a pooling layer and at least one convolutional layer.

[0074] like Figure 11 As shown, the use of the target classification model to process the pre-processed image to be classified also includes inputting the target obtained by the target detection network into the target classification network for target classification, and the inputting the target obtained by the target detection network into the target classification network for target classification includes the following steps:

[0075] Step S521: extracting features of the target acquired by the target detection network through a common convolution module and a Gabor convolution module respectively;

[0076] Step S522: concatenating the feature map output by the common convolution module and the feature map output by the Gabor convolution module to obtain a new feature map;

[0077] Step S523: performing a dimension-up operation on the new feature map;

[0078] Step S524: adding the dimension-upgraded feature map to the target obtained by the target detection network through a summation operation;

[0079] Step S525: The results obtained by the summation operation are aggregated through a global average pooling layer, and the classification task is completed through a softmax classifier.

[0080] like Figure 12 As shown, the display of the target classification results includes displaying the target classification results through a display module and sending the target classification results to a remote terminal for display. The display module includes an LCD touch screen, a physical button and a power indicator light. The physical button and the power indicator light are both connected to the LCD touch screen. The physical button is used to turn on and off the LCD touch screen, and the power indicator light is used to indicate the power connection status of the LCD touch screen.

[0081] In this embodiment, the power indicator light is a dual-color LED indicator light. The dual-color LED indicator light is green when the LCD touch screen is connected to a power source, and is red when the LCD touch screen is disconnected from a power source.

[0082] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A target classification method for complex scenes, characterized by: include: Collect images to be trained and images to be classified; Preprocess the collected images; Use the preprocessed images to be trained to create a dataset; Train the target classification model; Use the target classification model to process the preprocessed image to be classified; Display the target classification results; The use of the target classification model to process the preprocessed image to be classified includes inputting the preprocessed image to be classified into a target detection network for target detection, and the inputting the preprocessed image to be classified into the target detection network for target detection includes the following steps: extracting features from the preprocessed image to be classified by a feature extraction module to obtain a feature map; performing transposed convolution on the lower layer of the feature extraction module, and calculating the resolution of the low-layer feature map obtained by the transposed convolution; repeating the transposed convolution process until the resolution of the low-layer feature map of the feature extraction module is consistent with the resolution of the high-layer feature map of the feature extraction module; fusing the high-layer feature map and the low-layer feature map by a fusion convolution module; inputting the feature map output by the fusion convolution module into a convolution predictor for prediction; and selecting the best prediction result by a maximum suppression algorithm, wherein the feature extraction module includes multiple pairs of a pooling layer and at least one convolution layer; The use of the target classification model to process the pre-processed image to be classified also includes inputting the target obtained through the target detection network into the target classification network for target classification, and the inputting the target obtained through the target detection network into the target classification network for target classification includes the following steps: respectively extracting features of the target obtained by the target detection network through an ordinary convolution module and a Gabor convolution module; splicing the feature map output by the ordinary convolution module and the feature map output by the Gabor convolution module to obtain a new feature map; performing a dimensionality increase operation on the new feature map; adding the feature map after dimensionality increase and the target obtained by the target detection network through a summation operation; summing the results obtained by the summation operation through a global average pooling layer, and completing the classification task through a softmax classifier.

2. The target classification method for complex scenes according to claim 1, characterized in that: The collection of images to be trained includes the following steps: crawling image information containing complex scenes from multiple websites through a crawling module; managing the crawling process through a crawler management module; storing the crawled image information through a storage module, the crawling module includes a downloading unit, a parsing unit, a memory unit and a database unit, the downloading unit is used to download the web page to be crawled, the parsing unit is connected to the downloading unit, the parsing unit is used to parse out useful data from the web page downloaded by the downloading unit and save it into an object, the memory unit is connected to the parsing unit, the memory unit is used to persist the object parsed by the parsing unit, the database unit is connected to the memory unit, and the database unit is used to save the data processed by the memory unit; the crawler management module includes a status display unit and a crawler control unit, the status display unit is used to display the specific status of the crawling process, the crawler control unit is used to control the start, pause and crawling interval of the crawler; the storage module includes a computer hard disk.

3. The target classification method for complex scenes according to claim 1, characterized in that: The collecting of images to be classified includes obtaining image information to be classified through a camera module and storing the obtained image information to be classified in a storage module. The camera module includes a CMOS sensor, an electric zoom lens, a zoom drive motor, an LED light group, a relay, a brightness sensor, an infrared sensor and a pan-tilt head. The CMOS sensor, the electric zoom lens, the zoom drive motor, the LED light group, the relay, the brightness sensor and the infrared sensor are all installed on the pan-tilt head. The CMOS sensor is used to collect image information. The electric zoom lens is connected to the CMOS sensor. The zoom drive motor is used to drive the electric zoom lens to zoom. The relay is connected to the LED light group. The relay is used to drive the LED light group to illuminate. The brightness sensor is used to obtain ambient brightness information. The infrared sensor is used to detect whether there is a moving target in the monitoring area.

4. The target classification method for complex scenes according to claim 1, characterized in that: The preprocessing of the collected images includes preprocessing the collected images to be trained, and the preprocessing of the collected images to be trained includes the following steps: normalizing the collected images to be trained to have the same specifications; setting a feature point threshold, obtaining the number of feature points of the images to be trained, and deleting the images to be trained whose number of feature points is greater than the feature point threshold; performing feature binarization on the remaining images to be trained after deletion using a fuzzy mean binarization algorithm to separate the target to be detected from the background; calculating the ratio of the target to be detected to the entire image, and deleting the images to be trained whose ratio of the target to be detected to the entire image is less than a preset ratio value.

5. The target classification method for complex scenes according to claim 4, characterized in that: The preprocessing of the collected images also includes preprocessing the collected images to be classified, and the preprocessing of the collected images to be classified includes the following steps: performing distortion correction processing on the images to be classified through a convolutional neural network algorithm; performing noise reduction processing on the images to be classified after the distortion correction processing through a median filtering algorithm; and performing image grayscale processing on the images to be classified after the noise reduction processing.

6. The target classification method for complex scenes according to claim 1, characterized in that: The method of using the preprocessed images to be trained to produce a data set includes the following steps: performing a reduction operation on the preprocessed images to be trained using the resize function of the OpenCV library; dividing the images to be trained after the reduction operation into two parts according to a preset ratio, using one part as a training data set and the other part as a test data set; packaging the training data set into a training TFRecords picture set, and packaging the test data set into a test TFRecords picture set.

7. The target classification method for complex scenes according to claim 1, characterized in that: The training target classification model includes the following steps: training a target detection network using an SSD algorithm and a data set made using preprocessed images to be trained; training a target classification network using a convolutional neural network algorithm and a data set made using preprocessed images to be trained.

8. The target classification method for complex scenes according to claim 1, characterized in that: The display of the target classification results includes displaying the target classification results through a display module and sending the target classification results to a remote terminal for display. The display module includes an LCD touch screen, a physical button and a power indicator light. The physical button and the power indicator light are both connected to the LCD touch screen. The physical button is used to turn on and off the LCD touch screen, and the power indicator light is used to indicate the power connection status of the LCD touch screen.

Images