A feature extraction model construction and garbage detection method and device

Abstract

The present invention discloses a method and device for constructing a feature extraction model and detecting garbage. The method for constructing a feature extraction model comprises: establishing a garbage exposure target detection data set according to first garbage image data acquired; using the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; using the garbage target detection model to detect second garbage image data to obtain garbage targets and non-garbage targets; classifying the garbage targets and non-garbage targets and merging the classification results to establish a classification data set; using the classification data set to train a preset classification model to obtain a feature extraction model, wherein the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage detection, and in particular to a feature extraction model construction and garbage detection method and device. Background Art

[0002] With the development of cities and the improvement of people's quality of life, garbage has been increasingly "controlled". How to detect whether there is exposed garbage on the ground has become a difficult task. In the existing technology, the location of garbage accumulation is generally determined by manually checking video surveillance and manual inspections, and then the corresponding grid workers and cleaners are assigned to deal with it. A lot of manpower is consumed in the process of checking ground garbage. Summary of the invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defect in the prior art that checking ground garbage consumes a lot of manpower, thereby providing a feature extraction model construction and garbage detection method and device.

[0004] According to a first aspect, an embodiment of the present invention discloses a method for constructing a feature extraction model, including: establishing a garbage exposure target detection data set based on acquired first garbage image data; using the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; using the garbage target detection model to detect second garbage image data to obtain garbage targets and non-garbage targets; classifying the garbage targets and non-garbage targets and merging the classification results to establish a classification data set; using the classification data set to train a preset classification model to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set.

[0005] Optionally, a human and vehicle non-target detection data set is established based on the acquired human and vehicle non-image data; and the human and vehicle non-target detection data set is used to train a second preset model to obtain a human and vehicle non-target detection model.

[0006] Optionally, the method of establishing a garbage exposed target detection dataset based on the acquired first garbage image data includes: acquiring surveillance video data of a street and determining video data containing garbage scenes from the video; extracting frames of the video data containing garbage scenes to obtain image data and performing deduplication processing to obtain the first garbage image data; marking garbage targets in the first garbage image data and establishing a garbage exposed target detection dataset based on the marked garbage targets.

[0007] According to the second aspect, an embodiment of the present invention further discloses a garbage detection method, including: using a pre-established garbage target detection model and a human and vehicle non-target detection model to detect an image to be detected to determine garbage target information and human and vehicle non-target garbage information; filtering garbage targets located on human and vehicle non-targets to obtain first detected garbage target information according to first position information corresponding to the garbage target information and second position information corresponding to the human and vehicle non-target garbage information; cropping the first detected garbage target according to the first detected garbage target information and extracting a feature vector of the first detected garbage target using a feature extraction model constructed by the feature extraction model construction method described in the first aspect or any optional embodiment of the first aspect to obtain a target feature vector; comparing the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; filtering the non-garbage targets in the first detected garbage target according to the attribute information to obtain second detected garbage target information.

[0008] Optionally, after filtering the non-junk targets in the first detected junk targets according to the attribute information to obtain the second detected junk target information, the method also includes: storing the feature vector corresponding to the second detected junk target information in a preset database; when the second detected junk target is determined to be junk after review, updating the attribute information of the second detected junk target in the preset database.

[0009] Optionally, comparing the target feature vector with data in a preset database to determine attribute information corresponding to a first detected garbage target includes: determining K nearest neighbors corresponding to the first detected garbage target based on the feature vector corresponding to the first detected garbage target; querying attribute information corresponding to the K nearest neighbors based on the K nearest neighbors corresponding to the first detected garbage target; and determining attribute information corresponding to a detected garbage target based on the attribute information corresponding to the K nearest neighbors.

[0010] Optionally, filtering the non-junk targets in the first detected junk target according to the attribute information to obtain the second detected junk target information includes: determining the distance between the K nearest neighbors corresponding to the first detected junk target and the feature vector corresponding to the first detected junk target according to the feature vector corresponding to the first detected junk target; calculating the weight of the junk attribute corresponding to the first detected junk target according to the distance between the K nearest neighbors corresponding to the first detected junk target and the feature vector corresponding to the first detected junk target; determining the non-junk targets in the first detected junk target according to the weight of the junk attribute corresponding to the first detected junk target and filtering them to obtain the second detected junk target information.

[0011] According to the third aspect, an embodiment of the present invention further discloses a feature extraction model construction device, comprising: an establishment module, used to establish a garbage exposure target detection data set based on the acquired first garbage image data; a training module, used to use the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; a detection module, used to use the garbage target detection model to detect the second garbage image data to obtain garbage targets and non-garbage targets; a classification module, used to classify the garbage targets and non-garbage targets and merge the classification results to establish a classification data set; a first determination module, used to use the classification data set to train a preset classification model to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set.

[0012] According to a fourth aspect, an embodiment of the present invention further discloses a garbage detection device, comprising: a second determination module, used to detect an image to be detected using a pre-established garbage target detection model and a human and vehicle non-target detection model to determine garbage target information and human and vehicle non-target garbage information; a first filtering module, used to filter garbage targets located on human and vehicle non-targets according to first position information corresponding to the garbage target information and second position information corresponding to the human and vehicle non-target garbage information to obtain first detected garbage target information; an extraction module, used to crop the first detected garbage target according to the first detected garbage target information and extract a feature vector of the first detected garbage target using a feature extraction model constructed by the feature extraction model construction method described in the first aspect or any optional implementation method of the first aspect to obtain a target feature vector; a comparison module, used to compare the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; a second filtering module, used to filter the non-garbage targets in the first detected garbage target according to the attribute information to obtain second detected garbage target information.

[0013] According to the fifth aspect, an embodiment of the present invention also discloses an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the feature extraction model construction method as described in the first aspect or any optional embodiment of the first aspect, or to perform the steps of the garbage detection method as described in the second aspect or any optional embodiment of the second aspect.

[0014] The technical solution of the present invention has the following advantages:

[0015] The feature extraction model construction method / device provided by the present invention includes: establishing a garbage exposure target detection data set according to the acquired first garbage image data; using the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; using the garbage target detection model to detect the second garbage image data to obtain garbage targets and non-garbage targets; classifying the garbage targets and non-garbage targets and merging the classification results to establish a classification data set; using the classification data set to train a preset classification model to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set. The method of the present invention first collects garbage images of actual scenes to establish a garbage exposure target detection data set, then uses the garbage exposure target detection data set to train the first preset model to obtain a garbage target detection model, then uses the garbage target detection model to detect garbage targets and non-garbage targets on the second garbage image, constructs a classification data set of garbage targets and non-garbage targets, and establishes a feature extraction model according to the classification data set. The feature extraction model obtained by this method has a high accuracy rate and can meet the construction of feature extraction models in various scenarios to improve the accuracy of subsequent garbage detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a specific example of a method for constructing a feature extraction model in an embodiment of the present invention;

[0018] Figure 2 is a flowchart of a specific example of a garbage detection method in an embodiment of the present invention;

[0019] Figure 3 A principle block diagram of a specific example of a feature extraction model building device in an embodiment of the present invention;

[0020] Figure 4 is a principle block diagram of a specific example of a garbage detection device in an embodiment of the present invention;

[0021] Figure 5 FIG. 4 is a specific example diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0024] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] The embodiment of the present invention discloses a method for constructing a feature extraction model. Figure 1 As shown, the method comprises the following steps:

[0027] Step 101: Establish a garbage exposure target detection data set according to the acquired first garbage image data.

[0028] Exemplarily, the first image data may be image data containing junk information. Specifically, the first junk image data is acquired and a junk exposure target detection data set is established according to the junk images therein.

[0029] Step 102: Use the garbage exposure target detection dataset to train a first preset model to obtain a garbage target detection model. Exemplarily, the first preset model may be a deep learning algorithm model. Specifically, the first preset model includes but is not limited to a YOLOv5x model.

[0030] Step 103: Detect the second junk image data using the junk object detection model to obtain junk objects and non-junk objects.

[0031] Exemplarily, the second image data may be image data containing junk information. Specifically, a junk target detection model is used to detect junk targets and non-junk targets, each junk target and non-junk target in the second image data is labeled, and the labeled result is cropped to save each junk target and non-junk target in the image.

[0032] Step 104: classify the garbage objects and non-garbage objects and merge the classification results to establish a classification data set.

[0033] Specifically, garbage targets and non-garbage targets are classified. Garbage targets include but are not limited to paper scraps, cardboard, plastic garbage bags, bagged garbage, snakeskin bags, cartons, and scattered garbage. Non-garbage includes but is not limited to cameras, benches, carts, warning signs, and drain pipes. The classification results of garbage targets and non-garbage targets are merged to establish a classification data set.

[0034] Step 105: Use the classification data set to train a preset classification model to obtain a feature extraction model, where the feature extraction model is used to extract feature vectors of junk objects and non-junk objects in the classification data set.

[0035] Exemplarily, the preset classification model may include but is not limited to the EfficientNet classification model. Specifically, the EfficientNet classification model is trained using a classification data set, and the input is cropped garbage pictures and non-garbage pictures, and the output categories include but are not limited to paper scraps, cardboard, garbage bags, bagged garbage, snakeskin bags, cartons, scattered garbage, cameras, benches, carts, warning signs, and drainage. After the EfficientNet classification model is trained, the trained model is used to input pictures, and each picture outputs 1536-dimensional features, so that the picture is converted into a feature vector, and the feature vector is used for subsequent retrieval.

[0036] The feature extraction model construction method provided by the present invention includes: establishing a garbage exposure target detection data set according to the acquired first garbage image data; using the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; using the garbage target detection model to detect the second garbage image data to obtain garbage targets and non-garbage targets; classifying the garbage targets and non-garbage targets and merging the classification results to establish a classification data set; using the classification data set to train a preset classification model to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set. The method of the present invention first collects garbage images of actual scenes to establish a garbage exposure target detection data set, then uses the garbage exposure target detection data set to train the first preset model to obtain a garbage target detection model, and then uses the garbage target detection model to detect garbage targets and non-garbage targets on the second garbage image, constructs a classification data set of garbage targets and non-garbage targets, and establishes a feature extraction model according to the classification data set. The feature extraction model obtained by this method has high accuracy and can meet the construction of feature extraction models in various scenarios.

[0037] As an optional embodiment of the present invention, the method also includes: establishing a human and vehicle non-target detection data set based on the acquired human and vehicle non-image data; and using the human and vehicle non-target detection data set to train a second preset model to obtain a human and vehicle non-target detection model.

[0038] Exemplarily, the non-image data of people and vehicles may be image data containing non-garbage target information of people and vehicles. Specifically, the non-image data of people and vehicles are acquired and a garbage exposed target detection dataset is established based on the non-target images of people and vehicles therein, and the non-targets of people and vehicles may include but are not limited to pedestrians, motor vehicles, and non-motor vehicles.

[0039] As an optional embodiment of the present invention, the method of establishing a garbage exposed target detection data set based on the acquired first garbage image data includes: acquiring surveillance video data of a street and determining video data of garbage scenes from the video; extracting frames of the video data of the garbage scenes to obtain image data and performing deduplication processing to obtain the first garbage image data; marking garbage targets in the first garbage image data and establishing a garbage exposed target detection data set based on the marked garbage targets.

[0040] Exemplarily, the surveillance video data received from the video data is obtained and scenes with garbage are selected from them, the video scene data with garbage is subjected to frame extraction processing and repeated images are removed, the garbage targets in the images are labeled, and a garbage exposed target detection data set is established based on the garbage target labeling results. Specifically, garbage targets such as paper scraps, cardboard, plastic garbage bags, bagged packaged garbage, snakeskin bags, cartons, and scattered garbage on the image to be processed are labeled to create a garbage exposed target detection data set. This specific implementation method can also be applied to the establishment of a human and vehicle non-target detection data set.

[0041] The embodiment of the present invention also discloses a garbage detection method, such as Figure 2 As shown, the method comprises the following steps:

[0042] Step 201, using a pre-established garbage target detection model and a human and vehicle non-target detection model to detect the image to be detected to determine garbage target information and human and vehicle non-target garbage information;

[0043] Exemplarily, the garbage target detection model may include but is not limited to the garbage target information obtained in step 102 of the feature extraction model construction method, and the non-target detection model for people and vehicles may include but is not limited to the non-target garbage information for people and vehicles determined in the optional embodiment of the feature extraction model construction method. Specifically, an image to be detected is input, and the garbage on the image is detected using the garbage target detection model, and the upper left and lower right vertex coordinates (x1, y1), (x2, y2) of the rectangular box of each garbage target on the image are output, and the coordinates are the coordinates in the current image pixel coordinate system, and the non-target detection model for people and vehicles is used to detect the image to be detected, and the upper left and lower right vertex coordinates of the rectangular box of each non-target person and vehicle on the image are output.

[0044] Step 202: According to the first position information corresponding to the junk target information and the second position information corresponding to the human and vehicle non-target junk information, the junk targets located on the human and vehicle non-targets are filtered to obtain first detected junk target information.

[0045] Exemplarily, when it is determined that the garbage target is within the coverage of the non-targets of people and vehicles through the first position information corresponding to the garbage target information and the second position information corresponding to the non-target garbage information of people and vehicles, it is filtered. Specifically, the upper left and lower right vertex coordinates (x1, y1), (x2, y2) of the rectangular box of each garbage target are used to calculate the center coordinates of the garbage target x=(x1+x2) / 2, y=(y1+y2) / 2. According to the upper left and lower right vertex coordinates of the rectangular box of each non-target of people and vehicles, it is determined whether the center coordinates (x, y) of the detected garbage are within the rectangular box of the non-targets of people and vehicles. If yes, the garbage targets located on the non-targets of people and vehicles are filtered out to avoid misdetection of items on the non-targets of people and vehicles as garbage.

[0046] Step 203: The first detected garbage target is cropped according to the first detected garbage target information and a feature extraction model constructed by the feature extraction model construction method is used to extract a feature vector of the first detected garbage target to obtain a target feature vector.

[0047] Exemplarily, each of the first detected junk objects is cropped, and feature extraction is performed on the cropped junk object image according to the feature extraction model in the above embodiment to determine a target feature vector.

[0048] Step 204, compare the target feature vector with the data in a preset database to determine the attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores the relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set.

[0049] Exemplarily, the preset database may include but is not limited to a MongoDB database, in which the attribute information and feature vector relationship index of each garbage target and non-garbage target in the garbage classification data set are pre-stored. Specifically, the attribute information of each garbage and non-garbage target in the garbage classification data set stored in the MongoDB database includes but is not limited to the number, feature vector, path, category, whether it is garbage, and whether it has been manually reviewed. Faiss is used to index the feature vectors stored in the MongoDB garbage database, and the target feature vector is retrieved using the established Faiss index to determine the attribute information corresponding to the target feature vector. The feature vectors of each garbage target and non-garbage target in the preset database can be extracted in advance by the feature extraction model trained in the above embodiment.

[0050] Step 205: Filter the non-junk targets in the first detected junk targets according to the attribute information to obtain the second detected junk target information. Exemplarily, the attribute information corresponding to the target feature vector includes but is not limited to the number, feature vector, path, category, whether it is junk, and whether it has been manually reviewed corresponding to the first detected junk target. According to the attribute information corresponding to the target feature vector, it is determined whether the first detected junk target is junk or non-junk, and the non-junk targets in the first detected junk targets are filtered to obtain the second detected junk target information, and the second detected junk target is the current junk detection result.

[0051] The garbage detection method provided by the present invention includes: using a pre-established garbage target detection model and a human and vehicle non-target detection model to detect an image to be detected to determine garbage target information and human and vehicle non-target garbage information; filtering garbage targets located on human and vehicle non-targets to obtain first detected garbage target information according to first position information corresponding to the garbage target information and second position information corresponding to the human and vehicle non-target garbage information; cropping the first detected garbage target according to the first detected garbage target information and using the feature extraction model constructed by the feature extraction model construction method to extract feature vectors from the cropping results of the first detected garbage target to obtain a target feature vector; comparing the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; filtering the non-garbage targets in the first detected garbage target according to the attribute information to obtain second detected garbage target information. The method of the present invention uses a pre-established model to detect an image to be detected to determine garbage target information and non-target garbage information of people and vehicles, filters garbage targets located on non-targets of people and vehicles according to first position information corresponding to the garbage target information and second position information corresponding to the non-target garbage information of people and vehicles to obtain a first detected garbage target, crops the first detected garbage target and uses a feature extraction model constructed by a feature extraction model construction method to extract a feature vector for the first detected garbage target, uses a pre-established database to retrieve the feature vector to determine its attribute information and filters non-garbage in the first detected garbage target according to the attribute information to obtain a second detected garbage target. The adopted secondary filtering scheme can realize fast, efficient and accurate detection of garbage targets, can be applied to garbage detection in various scenarios with video surveillance, and has high portability.

[0052] As an optional implementation of the present invention, after filtering the non-junk targets in the first detected junk targets according to the attribute information to obtain the second detected junk target information, the method further includes: storing the feature vector corresponding to the second detected junk target information in a preset database; when the second detected junk target is determined to be junk after review, updating the attribute information of the second detected junk target in the preset database. Specifically, the feature vector corresponding to the second detected junk is stored in a MongoDB database, and its attribute information in the database is updated after manual review to determine whether it is junk.

[0053] As an optional embodiment of the present invention, the target feature vector is compared with data in a preset database to determine attribute information corresponding to the first detected garbage target, including: determining the K nearest neighbors corresponding to the first detected garbage target based on the feature vector corresponding to the first detected garbage target; querying the attribute information corresponding to the K nearest neighbors based on the K nearest neighbors corresponding to the first detected garbage target; and determining the attribute information corresponding to the first detected garbage target based on the attribute information corresponding to the K nearest neighbors.

[0054] Specifically, the established Faiss index is used to retrieve the target feature vector, and the K nearest neighbors of the first detected garbage target are output. According to the numbers of the K nearest neighbors retrieved by Faiss, the corresponding category, whether it is garbage and other attribute information are queried. The attribute information corresponding to the K nearest neighbors can be used as the attribute information corresponding to the first detected garbage target.

[0055] As an optional embodiment of the present invention, the filtering of non-junk targets in the first detected garbage target according to the attribute information to obtain second detected garbage target information includes: determining the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target according to the feature vector corresponding to the first detected garbage target; calculating the weight of the garbage attribute corresponding to the first detected garbage target according to the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target; determining the non-junk targets in the first detected garbage target according to the weight of the garbage attribute corresponding to the first detected garbage target and filtering them to obtain second detected garbage target information.

[0056] Specifically, the established Faiss index is used to retrieve the target feature vector, and the K nearest neighbors of the first detected junk target and the distance D between the target feature vector and the K nearest neighbors are output. The weight is calculated based on the distance D between the target feature vector and the K nearest neighbors, W = exp(-λD), where λ is 1. The weighted value is determined based on the junk attribute of the K nearest neighbors to determine whether the current target is junk. If the weight of the non-junk K nearest neighbors in the current target retrieval result is larger, the target will be filtered out.

[0057] The embodiment of the present invention also discloses a feature extraction model construction device, such as Figure 3As shown, the device includes: an establishment module 301, which is used to establish a garbage exposure target detection data set according to the acquired first garbage image data; a training module 302, which is used to use the garbage exposure target detection data set to train a first preset model to obtain a garbage target detection model; a detection module 303, which is used to use the garbage target detection model to detect the second garbage image data to obtain garbage targets and non-garbage targets; a classification module 304, which is used to classify the garbage targets and non-garbage targets and merge the classification results to establish a classification data set; a first determination module 305, which is used to use the classification data set to train a preset classification model to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set.

[0058] The feature extraction model construction device provided by the present invention comprises: a first establishment module, which is used to establish a garbage exposure target detection data set according to the acquired first garbage image data; a first training module, which is used to train a first preset model using the garbage exposure target detection data set to obtain a garbage target detection model; a detection module, which is used to detect the second garbage image data using the garbage target detection model to obtain garbage targets and non-garbage targets; a classification module, which is used to classify the garbage targets and non-garbage targets and merge the classification results to establish a classification data set; a first determination module, which is used to train a preset classification model using the classification data set to obtain a feature extraction model, and the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set. The device of the present invention first collects garbage images of actual scenes to establish a garbage exposure target detection data set, then uses the garbage exposure target detection data set to train the first preset model to obtain a garbage target detection model, then uses the garbage target detection model to detect garbage targets and non-garbage targets on the second garbage image, constructs a classification data set of garbage targets and non-garbage targets, and establishes a feature extraction model according to the classification data set. The feature extraction model obtained by this method has high accuracy and can meet the construction of feature extraction models in various scenarios.

[0059] As an optional embodiment of the present invention, the device also includes: a second establishment module, used to establish a human and vehicle non-target detection data set based on the acquired human and vehicle non-image data; a second training module, used to use the human and vehicle non-target detection data set to train a second preset model to obtain a human and vehicle non-target detection model.

[0060] As an optional embodiment of the present invention, the first establishment module includes: an acquisition module, used to acquire surveillance video data of a street and determine video data containing garbage scenes from the video; a deduplication module, used to extract frames of the video data containing garbage scenes to obtain image data and perform deduplication processing to obtain first garbage image data; and a labeling module, used to label garbage targets in the first garbage image data and establish a garbage exposure target detection data set based on the labeled garbage targets.

[0061] The embodiment of the present invention also discloses a garbage detection device, such as Figure 4 As shown, the device includes: a second determination module 501, which is used to detect the image to be detected by using a pre-established garbage target detection model and a human and vehicle non-target detection model to determine garbage target information and human and vehicle non-target garbage information; a first filtering module 502, which is used to filter the garbage targets located on the human and vehicle non-targets according to the first position information corresponding to the garbage target information and the second position information corresponding to the human and vehicle non-target garbage information to obtain the first detected garbage target information; an extraction module 503, which is used to crop the first detected garbage target according to the first detected garbage target information and extract the feature vector of the first detected garbage target using the feature extraction model constructed by the feature extraction model construction method to obtain the target feature vector; a comparison module 504, which is used to compare the target feature vector with the data in a preset database to determine the attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores the relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; a second filtering module 505, which is used to filter the non-garbage targets in the first detected garbage target according to the attribute information to obtain the second detected garbage target information.

[0062] The garbage detection device provided by the present invention includes: a second determination module, which is used to detect the image to be detected by using a pre-established garbage target detection model and a human and vehicle non-target detection model to determine garbage target information and human and vehicle non-target garbage information; a first filtering module, which is used to filter garbage targets located on human and vehicle non-targets according to first position information corresponding to the garbage target information and second position information corresponding to the human and vehicle non-target garbage information to obtain first detected garbage target information; an extraction module, which is used to crop the first detected garbage target according to the first detected garbage target information and extract feature vectors of the first detected garbage target using a feature extraction model constructed by a feature extraction model construction method to obtain a target feature vector; a comparison module, which is used to compare the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; and a second filtering module, which is used to filter non-garbage targets in the first detected garbage target according to the attribute information to obtain second detected garbage target information. The device of the present invention detects the image to be detected by using a pre-established model to determine garbage target information and non-target garbage information of people and vehicles, filters the garbage targets located on the non-targets of people and vehicles according to the first position information corresponding to the garbage target information and the second position information corresponding to the non-target garbage information of people and vehicles to obtain the first detected garbage target, crops the first detected garbage target and extracts the feature vector thereof by using the feature extraction model constructed by the feature extraction model construction method, retrieves the feature vector by using a pre-established database to determine its attribute information and filters the non-garbage in the first detected garbage target according to the attribute information to obtain the second detected garbage target. The adopted secondary filtering scheme can realize fast, efficient and accurate detection of garbage targets, can be applied to garbage detection in various scenarios with video surveillance, and has high portability.

[0063] As an optional embodiment of the present invention, the device also includes a storage module for storing the feature vector corresponding to the second detected garbage target information in a preset database; an update module for updating the attribute information of the second detected garbage target in the preset database when the second detected garbage target is determined to be garbage after review.

[0064] As an optional embodiment of the present invention, the comparison module includes: a third determination module, used to determine the K nearest neighbors corresponding to the first detected garbage target based on the feature vector corresponding to the first detected garbage target; a query module, used to query the attribute information corresponding to the K nearest neighbors based on the K nearest neighbors corresponding to the first detected garbage target; and a fourth determination module, used to determine the attribute information corresponding to the first detected garbage target based on the attribute information corresponding to the K nearest neighbors.

[0065] As an optional embodiment of the present invention, the second filtering module includes: a fifth determination module, which is used to determine the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target based on the feature vector corresponding to the first detected garbage target; a calculation module, which is used to calculate the weight of the garbage attribute corresponding to the first detected garbage target according to the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target; a filtering submodule, which is used to determine the non-garbage targets in the first detected garbage target according to the weight of the garbage attribute corresponding to the first detected garbage target and filter to obtain the second detected garbage target information.

[0066] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, the electronic device may include a processor 401 and a memory 402, wherein the processor 401 and the memory 402 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0067] The processor 401 may be a central processing unit (CPU). The processor 401 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0068] The memory 402 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the garbage detection method in the embodiment of the present invention. The processor 401 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 402, that is, the feature extraction model construction method or the garbage detection method in the above method embodiment is implemented.

[0069] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 401, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely arranged relative to the processor 401, and these remote memories may be connected to the processor 401 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0070] The one or more modules are stored in the memory 402, and when executed by the processor 401, the following is performed: Figure 1 The feature extraction model building method in the embodiment shown in the figure may be performed as follows: Figure 2 The garbage detection method in the illustrated embodiment.

[0071] For details of the above electronic equipment, please refer to Figure 1 The embodiment shown or Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0072] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0073] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A garbage detection method, characterized in that: include: Using the pre-established garbage target detection model and the human and vehicle non-target detection model to detect the image to be detected, the garbage target information and the human and vehicle non-target garbage information are determined; According to the first position information corresponding to the garbage target information and the second position information corresponding to the non-target garbage information of people and vehicles, the garbage targets located on the non-targets of people and vehicles are filtered to obtain the first detected garbage target information; The first detected garbage target is cropped according to the first detected garbage target information and a feature vector is extracted from the first detected garbage target using a feature extraction model to obtain a target feature vector; The feature extraction model is constructed by the following steps: Establishing a garbage exposed target detection data set according to the acquired first garbage image data; using the garbage exposed target detection data set to train a first preset model to obtain a garbage target detection model; Detecting the second garbage image data using the garbage target detection model to obtain garbage targets and non-garbage targets; classifying the garbage targets and non-garbage targets and merging the classification results to establish a classification data set; The classification data set is used to train a preset classification model to obtain a feature extraction model, wherein the feature extraction model is used to extract feature vectors of garbage targets and non-garbage targets in the classification data set; a human and vehicle non-target detection data set is established based on the acquired human and vehicle non-image data; Using the human and vehicle non-target detection data set to train a second preset model to obtain a human and vehicle non-target detection model; The method of establishing a garbage exposed target detection data set based on the acquired first garbage image data includes: acquiring surveillance video data of a street and determining video data of a garbage scene from the video; extracting frames of the video data of the garbage scene to obtain image data and performing deduplication processing to obtain the first garbage image data; marking garbage targets in the first garbage image data and establishing a garbage exposed target detection data set based on the marked garbage targets; Comparing the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the classification data set; According to the attribute information, non-junk objects in the first detected junk objects are filtered to obtain second detected junk object information.

2. The method according to claim 1, characterized in that: After filtering the non-junk targets in the first detected junk targets according to the attribute information to obtain the second detected junk target information, the method further includes: storing the feature vector corresponding to the second detected garbage target information in a preset database; When the second detected garbage target is determined to be garbage after review, the attribute information of the second detected garbage target is updated in the preset database.

3. The method according to claim 1, characterized in that Comparing the target feature vector with data in a preset database to determine attribute information corresponding to the first detected garbage target includes: Determine K nearest neighbors corresponding to the first detected garbage target according to the feature vector corresponding to the first detected garbage target; Querying attribute information corresponding to the K nearest neighbors according to the K nearest neighbors corresponding to the first detected garbage target; The attribute information corresponding to the first detected garbage target is determined according to the attribute information corresponding to the K nearest neighbors.

4. The method according to claim 1, characterized in that The filtering of non-junk targets in the first detected junk targets according to the attribute information to obtain second detected junk target information includes: Determining the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target according to the feature vector corresponding to the first detected garbage target; Calculate the weight of the garbage attribute corresponding to the first detected garbage target according to the distance between the K nearest neighbors corresponding to the first detected garbage target and the feature vector corresponding to the first detected garbage target; According to the weights of the garbage attributes corresponding to the first detected garbage targets, non-garbage targets in the first detected garbage targets are determined and filtered to obtain second detected garbage target information.

5. A garbage detection device, characterized in that: The method for executing claim 1 comprises: The second determination module is used to detect the image to be detected by using a pre-established garbage target detection model and a human and vehicle non-target detection model to determine garbage target information and human and vehicle non-target garbage information; A first filtering module is used to filter the garbage targets located on the non-targets of people and vehicles to obtain first detected garbage target information according to the first position information corresponding to the garbage target information and the second position information corresponding to the non-target garbage information of people and vehicles; An extraction module, used for cutting the first detected garbage target according to the first detected garbage target information and extracting a feature vector of the first detected garbage target using a feature extraction model to obtain a target feature vector; A comparison module, used to compare the target feature vector with data in a preset database to determine the attribute information corresponding to the first detected garbage target, wherein the preset database pre-stores a relationship index between the attribute information and the feature vector of each garbage target and non-garbage target in the garbage classification data set; The second filtering module is used to filter the non-junk targets in the first detected junk targets according to the attribute information to obtain second detected junk target information.

6. An electronic device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the garbage detection method as described in any one of claims 1-4.

Images