Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method. The method comprises the following steps: acquiring a garbage recognition result set; extracting first color data from the first garbage recognition box, and extracting second color data from the second garbage recognition box; based on the first color data and the second color data, calculating to obtain a color difference ratio between the first garbage recognition box and the second garbage recognition box; and judging whether the first road image and the second road image are rejected in the garbage recognition result set based on the color difference proportion. According to the embodiment of the invention, the color difference ratio is used as a judgment basis for judging whether the road image is removed or not, so that when the color difference ratio meets the removal condition, the first road image and the second road image with the abnormal color difference ratio can be removed, and the recognition accuracy of the garbage recognition result set is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] At present, urban road garbage recognition mainly relies on artificial intelligence (AI) technology, which generates recognition models for garbage detection by training a large number of garbage scene images. However, these existing technologies face many interference factors in practical applications, such as sunlight, shadows, and leaves, which can lead to a high system misjudgment rate, resulting in a garbage recognition result set with low recognition accuracy after recognition, which in turn affects the cleaning and management efficiency of urban roads. Therefore, how to provide an image processing method that can improve the recognition accuracy of the garbage recognition result set has become an urgent problem to be solved. Summary of the invention

[0003] The embodiment of the present invention provides an image processing method, which aims to solve the problem of low recognition accuracy of the existing garbage recognition result set. By extracting corresponding color data from the first garbage recognition frame and the second garbage recognition frame, and calculating the color difference ratio between the first garbage recognition frame and the second garbage recognition frame according to the corresponding color data, the color difference ratio can be used as a basis for judging whether to remove the road image, and then when the color difference ratio meets the removal condition, the first road image and the second road image with abnormal color difference ratio can be removed to improve the recognition accuracy of the garbage recognition result set.

[0004] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising the following steps:

[0005] Acquire a garbage identification result set, wherein the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image;

[0006] extracting first color data in the first garbage identification frame, and extracting second color data in the second garbage identification frame;

[0007] Based on the first color data and the second color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated;

[0008] Based on the color difference ratio, it is determined whether to exclude the first road image and the second road image from the garbage identification result set.

[0009] Optionally, before obtaining the garbage identification result set, the method further includes:

[0010] Acquire a road image sequence of a target road, wherein the road image sequence includes multiple frames of road images;

[0011] Inputting the plurality of frames of the road image into a preset garbage recognition model for garbage recognition processing to obtain a garbage recognition frame corresponding to each frame of the road image;

[0012] The garbage identification result set is constructed based on the road image and the corresponding garbage identification frame.

[0013] Optionally, constructing the garbage identification result set based on the road image and the corresponding garbage identification frame includes:

[0014] Extracting the first road image and the second road image from a plurality of frames of the road image;

[0015] Expanding the garbage identification frames corresponding to the first road image and the second road image according to a preset ratio to obtain the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image;

[0016] The garbage identification result set is constructed based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image.

[0017] Optionally, the first color data is composed of a first color code, the second color data is composed of a second color code, and the calculating, based on the first color data and the second color data, to obtain a color difference ratio between the first garbage identification frame and the second garbage identification frame includes:

[0018] When the first color code and the second color code do not belong to the preset color space, the first color code is subjected to color code conversion processing to obtain first target color data, and the second color code is subjected to color code conversion processing to obtain second target color data;

[0019] Based on the first target color data and the second target color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated.

[0020] Optionally, the calculating, based on the first target color data and the second target color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame includes:

[0021] Based on the first target color data and the second target color data, a color difference between each first pixel in the first garbage identification frame and each second pixel in the second garbage identification frame is calculated, and each first pixel corresponds to one second pixel;

[0022] After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference ratio is obtained.

[0023] Optionally, after the color differences between all the first pixels and the corresponding second pixels are calculated, obtaining the color difference ratio includes:

[0024] After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference between each of the first pixel points and the corresponding second pixel point is obtained;

[0025] Determine the first pixel point and the second pixel point corresponding to the color difference being greater than a preset color difference threshold as difference pixels;

[0026] Counting the number of the difference pixels and the total number of the first pixels and the second pixels;

[0027] The color difference ratio is calculated based on the number of the difference pixels and the total number.

[0028] Optionally, judging whether to exclude the first road image and the second road image from the garbage identification result set based on the color difference ratio includes:

[0029] When the color difference ratio is greater than a preset color difference ratio threshold, removing the first road image and the second road image from the garbage identification result set;

[0030] When the color difference ratio is less than a preset color difference ratio threshold, the first road image and the second road image are retained in the garbage identification result set.

[0031] In a second aspect, an embodiment of the present invention further provides an image processing device, the image processing device comprising:

[0032] A first acquisition module is used to acquire a garbage identification result set, wherein the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image;

[0033] a first extraction module, configured to extract first color data from the first garbage identification frame, and extract second color data from the second garbage identification frame;

[0034] A first calculation module, configured to calculate a color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first color data and the second color data;

[0035] The first judgment module is used to judge whether to exclude the first road image and the second road image from the garbage identification result set based on the color difference ratio.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method provided in the embodiment of the present invention when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method provided in the embodiment of the invention are implemented.

[0038] In an embodiment of the present invention, a garbage identification result set is obtained, and the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image; first color data is extracted from the first garbage identification frame, and second color data is extracted from the second garbage identification frame; based on the first color data and the second color data, the color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated; based on the color difference ratio, it is determined whether to remove the first road image and the second road image from the garbage identification result set. By extracting corresponding color data from the first garbage identification frame and the second garbage identification frame, and calculating the color difference ratio between the first garbage identification frame and the second garbage identification frame according to the corresponding color data, the color difference ratio can be used as a basis for determining whether to remove the road image, and then when the color difference ratio meets the removal condition, the first road image and the second road image with abnormal color difference ratio can be removed to improve the recognition accuracy of the garbage identification result set. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present invention;

[0041] Figure 2 is a structural schematic diagram of an image processing device provided in an embodiment of the present invention;

[0042] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, Figure 1 is a flowchart of an image processing method provided by an embodiment of the present invention, comprising:

[0045] 101. Obtain a garbage identification result set.

[0046] In an embodiment of the present invention, the above-mentioned image processing method can be applied to an image processing platform, and the above-mentioned image processing platform can be constructed by a server or a server cluster. The above-mentioned server or server cluster can be any electronic device with functions such as image recognition, image processing, image analysis, data transmission, and data storage.

[0047] After providing the garbage identification result set to the image processing platform, the image processing platform can implement any one of the image processing methods provided in the embodiments of the present invention to determine whether to remove the first road image and the corresponding second road image in the garbage identification result set, and obtain the target garbage identification result set after all the first road images and the corresponding second road images are determined. The recognition accuracy of the target garbage identification result set is higher than that of the garbage identification result set.

[0048] The garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image. It should be noted that the first road image may be multiple frames or one, and the second road image may be multiple frames or one. Each frame of the first road image corresponds to one second road image, wherein the relationship between the first road image and the second road image may be an adjacent relationship, a fixed time interval relationship or a random relationship.

[0049] Specifically, assume that the garbage identification result set includes road images A1, A2, A3 and A4, and A1, A2, A3 and A4 all have corresponding image shooting times, wherein the image shooting time of A1 is earlier than the image shooting time of A2, the image shooting time of A2 is earlier than the image shooting time of A3, and the image shooting time of A3 is earlier than the image shooting time of A4. The above-mentioned adjacent relationship can be adjacent in order in the garbage identification result set, or adjacent in shooting time. If it is an adjacent relationship, when A1 is the first road image, A2 is the second road image, when A2 is the first road image, A3 is the second road image, and when A3 is the first road image, A4 is the second road image. That is, adjacent road images are continuously selected as the first road image and the second road image in chronological order.

[0050] Alternatively, if the time interval is fixed, assuming that the fixed time interval is 10 seconds, the image capture time of A1 is 00:00:00, the image capture time of A2 is 00:00:05, the image capture time of A3 is 00:00:10, and the image capture time of A4 is 00:00:15, then when A1 is the first road image, A3 is the second road image, and when A2 is the first road image, A4 is the second road image. That is, road images that meet the fixed time interval are continuously selected in time order as the first road image and the second road image.

[0051] Alternatively, a road image in the garbage identification result set may be selected as the first road image and the corresponding second road image by random selection.

[0052] More specifically, the above-mentioned image processing platform may be deployed with a target detector, and the above-mentioned image processing platform may implement the above-mentioned image recognition function based on the above-mentioned target detector. The above-mentioned image processing platform may obtain a road image sequence of a target road through the above-mentioned data transmission function, and perform garbage recognition processing on each frame of the road image in the above-mentioned road image sequence through the above-mentioned target detector to obtain a garbage recognition frame of each frame of the road image. The above-mentioned garbage recognition result set is constructed based on the garbage recognition frame and the corresponding road image, or the user may manually upload any garbage recognition result set, so that the above-mentioned image processing platform implements an image processing method based on the above-mentioned garbage recognition result set to improve the recognition accuracy of the above-mentioned garbage recognition result set.

[0053] 102. Extract first color data from the first garbage identification frame, and extract second color data from the second garbage identification frame.

[0054] In an embodiment of the present invention, there is a corresponding relationship between the first garbage identification frame and the second garbage identification frame, that is, the image coordinates corresponding to the first garbage identification frame and the second garbage identification frame are the same. For example, assume that the first road image includes first garbage identification frames A1 and A2, and the second road image includes second garbage identification frames B1 and B2, wherein the image coordinates of A1 in the first road image are (15, 15), the image coordinates of A2 in the first road image are (30, 30), the image coordinates of B1 in the second road image are (15, 15), and the image coordinates of B2 in the second road image are (30, 30). Therefore, when A1 is used as the first garbage identification frame, the second garbage identification frame is B1, and when A2 is used as the first garbage identification frame, the second garbage identification frame is B2.

[0055] The first color data and the second color data are determined according to the color space of the first road image and the second road image. For example, when the color space corresponding to the first road image and the second road image is an RGB color space, the first color data may be the RGB value of each pixel in the first garbage identification frame, and the second color data may be the RGB value of each pixel in the second garbage identification frame. Or when the color space corresponding to the first road image and the second road image is an HSV color space, the first color data may be the HSV value of each pixel in the first garbage identification frame, and the second color data may be the HSV value of each pixel in the second garbage identification frame.

[0056] In the first garbage identification frame and the second garbage identification frame, first color data and second color data corresponding to the color space are extracted.

[0057] It should be noted that, in general, the color spaces of the first road image and the second road image are consistent, but when they are inconsistent, the color data corresponding to the color space can also be extracted, and in subsequent use, the color codes in the color data can be unified into the color codes of the same color space. It can be understood that the color data is composed of color codes.

[0058] 103. Based on the first color data and the second color data, calculate a color difference ratio between the first garbage identification frame and the second garbage identification frame.

[0059] In an embodiment of the present invention, when the first color code in the first color data and the second color code in the second color data belong to the RGB color space, the first color code and the second color code need to be converted into the HSV color space to obtain first target color data corresponding to the first color data and second target color data corresponding to the second color data.

[0060] After obtaining the first target color data and the second target color data, since the color codes corresponding to the first target color data and the second target color data belong to the HSV color space, the first target color data includes the HSV value of each first pixel in the first garbage identification frame, and the second target color data includes the HSV value of each second pixel in the second garbage identification frame.

[0061] Each first pixel corresponds to a second pixel, and the corresponding relationship is that the image coordinates in the image are the same. The difference in HSV value (i.e., color difference) between each first pixel and the corresponding second pixel is calculated, so that after calculating the difference in HSV value (i.e., color difference) between all first pixels and the corresponding second pixels, the number of pixels with differences is counted, and the above color difference ratio is calculated based on the ratio of the number of pixels with differences and the total number of first pixels and second pixels.

[0062] 104. Based on the color difference ratio, determine whether to exclude the first road image and the second road image from the garbage identification result set.

[0063] In the embodiment of the present invention, a color difference ratio threshold may be preset, and the color difference ratio threshold may be set according to user requirements or historical image processing experience, or may be obtained through a limited number of experiments.

[0064] When the color difference ratio is greater than a preset color difference ratio threshold, the first road image and the second road image are eliminated from the junk identification result set. Conversely, when the color difference ratio is less than or equal to the preset color difference ratio threshold, the first road image and the second road image are retained in the junk identification result set.

[0065] For example, when the color difference between the first garbage identification frame and the second garbage identification frame accounts for 40% and the preset color difference percentage threshold is 50%, the first road image and the second road image can be eliminated from the garbage identification result set.

[0066] It is understandable that the image coordinates of the first garbage identification frame and the second garbage identification frame are the same, and the above-mentioned image coordinates can be the vertex coordinates of the identification frame, for example, the upper left vertex coordinates, the upper right vertex coordinates, the lower left vertex coordinates or the lower right vertex coordinates. The road images in the same garbage identification result set are usually size-normalized, and their image sizes are usually the same, for example, they are all 1920*1080 resolution, or other resolutions, etc., and the road images in the same garbage identification result set are road images in the same road image sequence, and their shooting locations are the same, and only the shooting time is changed, so the first garbage identification frame and the second garbage identification frame with the same image coordinates can be selected to determine whether there is a color difference ratio between the two. Then determine whether to eliminate the first road image and the second road image in the garbage identification result set.

[0067] It should be noted that, under normal circumstances, if the color difference ratio is greater than the preset color difference ratio threshold, it means that there are misjudgments in the recognition results corresponding to the first road image and the second road image (i.e., the garbage recognition box). The possible reason is that the misjudgment is caused by light and shadow, leaves being blown by the wind, or light jitter. Therefore, the first road image and the second road image need to be eliminated at the same time.

[0068] In an embodiment of the present invention, a garbage identification result set is obtained, and the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image; first color data is extracted from the first garbage identification frame, and second color data is extracted from the second garbage identification frame; based on the first color data and the second color data, the color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated; based on the color difference ratio, it is determined whether to remove the first road image and the second road image from the garbage identification result set. By extracting corresponding color data from the first garbage identification frame and the second garbage identification frame, and calculating the color difference ratio between the first garbage identification frame and the second garbage identification frame according to the corresponding color data, the color difference ratio can be used as a basis for determining whether to remove the road image, and then when the color difference ratio meets the removal condition, the first road image and the second road image with abnormal color difference ratio can be removed to improve the recognition accuracy of the garbage identification result set.

[0069] It can be understood that in the specific implementation of the present application, related data such as the garbage identification result set, the first road image, the second road image, the road image sequence, etc., when the embodiments in the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data and the construction, training and use of related models need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0070] It should be noted that the image processing method provided in the embodiment of the present invention can be applied to devices such as computers and servers that can perform image processing.

[0071] Optionally, before the step of obtaining a garbage identification result set, a road image sequence of the target road may also be obtained, the road image sequence including multiple frames of road images; the multiple frames of road images are input into a preset garbage identification model for garbage identification processing to obtain a garbage identification frame corresponding to each frame of the road image; and a garbage identification result set is constructed based on the road images and the corresponding garbage identification frames.

[0072] In an embodiment of the present invention, the above-mentioned preset garbage recognition model (or can be understood as a target detector) can be constructed by a deep learning algorithm such as a convolutional neural network (CNN) or a YOLO series to obtain a garbage recognition model to be trained, and the above-mentioned garbage recognition model to be trained is trained for garbage recognition, so as to obtain the above-mentioned preset garbage recognition model.

[0073] Specifically, the training process of the preset garbage recognition model may include steps such as data collection, data annotation, and model training. The images and video data of the road scene are collected in real time by using the cameras and other equipment installed on the road. The collected images and video data are manually annotated to mark garbage and other objects. The annotated images and video data are used to train the garbage recognition model to be trained. After the training is completed, the preset garbage recognition model is obtained.

[0074] After obtaining the preset garbage recognition model, the preset garbage recognition model can be deployed on the image processing platform. When garbage recognition is required for a certain road, a road image sequence of the road can be obtained, and the road image sequence can be input into the preset garbage recognition model for garbage recognition processing to obtain a garbage recognition frame corresponding to each frame of the road image. Thus, a garbage recognition result set is constructed based on the road image and the corresponding garbage recognition frame.

[0075] Optionally, in the step of constructing a garbage identification result set based on road images and corresponding garbage identification frames, a first road image and a second road image can be extracted from multiple frames of road images; the garbage identification frames corresponding to the first road image and the second road image are expanded according to a preset ratio to obtain a first garbage identification frame corresponding to the first road image and a second garbage identification frame corresponding to the second road image; and a garbage identification result set is constructed based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image.

[0076] In the embodiment of the present invention, the extraction method of the first road image and the second road image depends on the corresponding relationship between the first road image and the second road image, that is, the adjacent relationship, the fixed time interval relationship or the random relationship.

[0077] After extracting the first road image and the second road image, the garbage identification frames corresponding to the first road image and the second road image can be expanded to obtain the first garbage identification frame and the second garbage identification frame. It can be understood that the first garbage identification frame and the second garbage identification frame are obtained by expanding the garbage identification result (i.e., the garbage identification frame) output by the preset garbage identification model. Among them, the preset ratio can be to expand the garbage identification frame to 10% to 20%, and the specific ratio can be set according to actual needs. Specifically, the center point of the garbage identification frame can be used as a reference, and the garbage identification frame can be expanded according to the preset ratio.

[0078] It should be noted that after the garbage identification frame is expanded, the number of pixels in the first garbage identification frame or the second garbage identification frame will be greater. In this way, after the garbage identification result set is constructed based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image, it can be ensured that when the color difference ratio is subsequently calculated, more areas that may be interfered with can be covered, thereby improving the accuracy of the color difference ratio.

[0079] In addition to the above-mentioned first road image, the second road image, the first garbage identification frame and the second garbage identification frame, the above-mentioned garbage identification result set may also include other road images in the above-mentioned multiple-frame road images (that is, not the first road image and not the second road image) and corresponding garbage identification frames (that is, garbage identification frames that undergo external expansion processing).

[0080] Specifically, after obtaining the above-mentioned first road image, second road image, first garbage identification frame and second garbage identification frame, the above-mentioned first road image, second road image, first garbage identification frame, second garbage identification frame and the above-mentioned other road images and garbage identification frames corresponding to other road images can be added to the preset recognition result empty set to obtain the above-mentioned garbage identification result set.

[0081] Optionally, in the step of calculating the color difference ratio between the first garbage identification box and the second garbage identification box based on the first color data and the second color data, when the first color code and the second color code do not belong to the preset color space, the first color code can be subjected to color code conversion processing to obtain first target color data, and the second color code can be subjected to color code conversion processing to obtain second target color data; based on the first target color data and the second target color data, the color difference ratio between the first garbage identification box and the second garbage identification box can be calculated.

[0082] In an embodiment of the present invention, the first color data is composed of a first color code, and the second color data is composed of a second color code. The preset color space may be an HSV color space, where H represents hue, S represents saturation, and V represents brightness.

[0083] For example, when the first color code and the second color code belong to the RGB color space, the first color code and the second color code do not belong to the HSV color space. At this time, the first color code and the second color code can be converted to the HSV color space through the OpenCV library of the Python programming language or other methods. Thereby obtaining the above-mentioned first target color data and the above-mentioned second target color data. It can be understood that, since the above-mentioned first road image and the second road image belong to the same road image sequence, and the same road image sequence is obtained by shooting with the same camera, the color codes corresponding to the first road image and the second road image usually belong to the same color space, so when performing the code conversion, it is necessary to simultaneously convert the first color code and the second color code to the preset color space.

[0084] Since color information (hue and saturation) is independent of brightness, color recognition becomes more reliable under complex lighting conditions, such as shadows, overexposure, or uneven lighting. Similarly, since the HSV color space includes the dimension of brightness, calculating the color difference ratio in the HSV color space can better reflect whether the first road image and the second road image are affected by light and shadow or leaves are affected by wind or light jitter, resulting in reduced accuracy of their recognition results (i.e., garbage recognition frames).

[0085] Optionally, in the step of calculating the color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first target color data and the second target color data, the color difference between each first pixel point in the first garbage identification frame and each second pixel point in the second garbage identification frame can also be calculated based on the first target color data and the second target color data; after the color difference between all first pixels and the corresponding second pixels is calculated, the color difference ratio is obtained.

[0086] In an embodiment of the present invention, each first pixel corresponds to a second pixel, and the corresponding relationship is the image coordinate correspondence. The above-mentioned first target color data and the second target color data are constructed by color coding of the HSV color space, and the above-mentioned color coding includes the HSV value of each pixel in the garbage identification frame, and the above-mentioned HSV value includes the H value (i.e. hue), S value (i.e. saturation) and V value (i.e. brightness). Therefore, the HSV value of each first pixel in the first garbage identification frame can be determined in the above-mentioned first target color data, and the HSV value of each second pixel in the second garbage identification frame can be determined in the second target color data, so as to calculate the color difference between each first pixel and the corresponding second pixel according to the HSV value of the above-mentioned first pixel and the HSV value of the corresponding second pixel.

[0087] Specifically, the above color difference can be further explained by the following formula:

[0088]

[0089] Among them, the above D (a1 , b1) represents the color difference between the first pixel a1 and the corresponding second pixel a2. a1 It is represented by the H value of the first pixel a1. b1 It is represented by the H value of the second pixel b1. a1 It is represented by the S value of the first pixel a1. b1 It is represented by the S value of the second pixel b1. a1 It is represented by the V value of the first pixel a1. b1 Represented as the V value of the second pixel b1.

[0090] After the color difference between all the first pixels and the corresponding second pixels is calculated, the number of the first pixels and the corresponding second pixels with color difference (i.e., D is not 0) is counted. Alternatively, the number of the first pixels and the corresponding second pixels with color difference greater than a preset color difference threshold value can be counted. The number of the first pixels and the corresponding second pixels with color difference (i.e., D is not 0) is counted. Alternatively, the ratio between the number of the first pixels and the corresponding second pixels with color difference greater than a preset color difference threshold value and the total number of all the first pixels and the corresponding second pixels can be counted as the above color difference proportion.

[0091] Optionally, in the step of obtaining the color difference percentage after the color difference between all first pixel points and the corresponding second pixel points is calculated, the color difference between each first pixel point and the corresponding second pixel point can be obtained after the color difference between all first pixel points and the corresponding second pixel points is calculated; the first pixel points and the second pixel points corresponding to the color difference greater than a preset color difference threshold are determined as difference pixels; the number of difference pixels and the total number of first pixel points and second pixel points are counted; and the color difference percentage is calculated based on the number of difference pixels and the total number.

[0092] In an embodiment of the present invention, the preset color difference threshold can be set based on historical image processing experience, or can be obtained through a limited number of experiments, or can be set based on actual needs, wherein the color difference threshold can be set to 0.5, that is, the first pixel point with a color difference greater than 0.5 and the corresponding second pixel point are difference pixels. After counting the number of difference pixels, the ratio of the number of difference pixels to the total number is determined as the color difference ratio.

[0093] Optionally, in the step of determining whether to eliminate the first road image and the second road image from the junk identification result set based on the color difference ratio, the first road image and the second road image may be eliminated from the junk identification result set when the color difference ratio is greater than a preset color difference ratio threshold; and the first road image and the second road image may be retained in the junk identification result set when the color difference ratio is less than the preset color difference ratio threshold.

[0094] In the embodiment of the present invention, the first road image and the second road image with abnormal color difference ratio (i.e., the color difference ratio is greater than a preset color difference ratio threshold) are removed, and the first road image and the second road image with normal color difference ratio are retained. This can effectively remove interference factors in road garbage recognition, thereby significantly improving the recognition accuracy of the garbage recognition result set.

[0095] like Figure 2As shown, an embodiment of the present invention further provides an image processing device, comprising:

[0096] A first acquisition module 201 is used to acquire a garbage recognition result set, wherein the garbage recognition result set includes a first road image and a first garbage recognition frame in the first road image, a second road image and a second garbage recognition frame in the second road image;

[0097] A first extraction module 202, configured to extract first color data from the first garbage identification frame, and to extract second color data from the second garbage identification frame;

[0098] A first calculation module 203, configured to calculate a color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first color data and the second color data;

[0099] The first judgment module 204 is configured to judge whether to exclude the first road image and the second road image from the garbage identification result set based on the color difference ratio.

[0100] Optionally, the image processing device further includes:

[0101] A second acquisition module is used to acquire a road image sequence of a target road, wherein the road image sequence includes multiple frames of road images;

[0102] A first recognition module, used for inputting a plurality of frames of the road image into a preset garbage recognition model for garbage recognition processing, and obtaining a garbage recognition frame corresponding to each frame of the road image;

[0103] The first construction module is used to construct the garbage identification result set based on the road image and the corresponding garbage identification frame.

[0104] Optionally, the first building block includes:

[0105] A first extraction submodule, configured to extract the first road image and the second road image from a plurality of frames of the road image;

[0106] A first expansion submodule, configured to expand the garbage identification frames corresponding to the first road image and the second road image according to a preset ratio to obtain the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image;

[0107] The first construction submodule is used to construct the garbage identification result set based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image.

[0108] Optionally, the first color data is composed of a first color code, the second color data is composed of a second color code, and the first calculation module 203 includes:

[0109] A first conversion submodule is used for, when the first color code and the second color code do not belong to a preset color space, performing a color code conversion process on the first color code to obtain first target color data, and performing a color code conversion process on the second color code to obtain second target color data;

[0110] The first calculation submodule is used to calculate a color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first target color data and the second target color data.

[0111] Optionally, the first calculation submodule includes:

[0112] a first calculating unit, configured to calculate a color difference between each first pixel in the first garbage identification frame and each second pixel in the second garbage identification frame based on the first target color data and the second target color data, wherein each first pixel corresponds to one second pixel;

[0113] The first processing unit is configured to obtain the color difference ratio after the color differences between all the first pixels and the corresponding second pixels are calculated.

[0114] Optionally, the first processing unit includes:

[0115] A first processing subunit is used to obtain the color difference between each of the first pixel points and the corresponding second pixel point after the color differences between all the first pixel points and the corresponding second pixel points are calculated;

[0116] A first determining subunit, configured to determine the first pixel point and the second pixel point corresponding to the color difference being greater than a preset color difference threshold as difference pixels;

[0117] A first counting subunit, configured to count the number of the difference pixels and the total number of the first pixels and the second pixels;

[0118] The first calculation subunit is used to calculate the color difference ratio based on the number of the difference pixels and the total number.

[0119] Optionally, the first determining module 204 includes:

[0120] A rejection submodule, configured to reject the first road image and the second road image from the garbage identification result set when the color difference ratio is greater than a preset color difference ratio threshold;

[0121] The retaining submodule is used to retain the first road image and the second road image in the garbage identification result set when the color difference ratio is less than a preset color difference ratio threshold.

[0122] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, characterized in that it includes a processor, and the processor can execute any one of the above-mentioned image processing methods.

[0123] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing the image processing method which is stored in the memory 302 and can be run on the processor 301, wherein:

[0124] The processor 301 runs the computer program of the image processing method stored in the memory 302 to perform the following steps:

[0125] Acquire a garbage identification result set, wherein the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image;

[0126] extracting first color data in the first garbage identification frame, and extracting second color data in the second garbage identification frame;

[0127] Based on the first color data and the second color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated;

[0128] Based on the color difference ratio, it is determined whether to exclude the first road image and the second road image from the garbage identification result set.

[0129] Optionally, before obtaining the garbage identification result set, the method executed by the processor 301 further includes:

[0130] Acquire a road image sequence of a target road, wherein the road image sequence includes multiple frames of road images;

[0131] Inputting the plurality of frames of the road image into a preset garbage recognition model for garbage recognition processing to obtain a garbage recognition frame corresponding to each frame of the road image;

[0132] The garbage identification result set is constructed based on the road image and the corresponding garbage identification frame.

[0133] Optionally, the step of constructing the garbage identification result set based on the road image and the corresponding garbage identification frame performed by the processor 301 includes:

[0134] Extracting the first road image and the second road image from a plurality of frames of the road image;

[0135] Expanding the garbage identification frames corresponding to the first road image and the second road image according to a preset ratio to obtain the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image;

[0136] The garbage identification result set is constructed based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image.

[0137] Optionally, the first color data is composed of a first color code, and the second color data is composed of a second color code, and the processor 301 calculates the color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first color data and the second color data, including:

[0138] When the first color code and the second color code do not belong to the preset color space, the first color code is subjected to color code conversion processing to obtain first target color data, and the second color code is subjected to color code conversion processing to obtain second target color data;

[0139] Based on the first target color data and the second target color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated.

[0140] Optionally, the processor 301 calculates the color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first target color data and the second target color data, including:

[0141] Based on the first target color data and the second target color data, a color difference between each first pixel in the first garbage identification frame and each second pixel in the second garbage identification frame is calculated, and each first pixel corresponds to one second pixel;

[0142] After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference ratio is obtained.

[0143] Optionally, after the color difference calculation between all the first pixels and the corresponding second pixels is completed, the processor 301 obtains the color difference ratio, including:

[0144] After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference between each of the first pixel points and the corresponding second pixel point is obtained;

[0145] Determine the first pixel point and the second pixel point corresponding to the color difference being greater than a preset color difference threshold as difference pixels;

[0146] Counting the number of the difference pixels and the total number of the first pixels and the second pixels;

[0147] The color difference ratio is calculated based on the number of the difference pixels and the total number.

[0148] Optionally, the determining, based on the color difference ratio, whether to exclude the first road image and the second road image from the garbage identification result set performed by the processor 301 includes:

[0149] When the color difference ratio is greater than a preset color difference ratio threshold, removing the first road image and the second road image from the garbage identification result set;

[0150] When the color difference ratio is less than a preset color difference ratio threshold, the first road image and the second road image are retained in the garbage identification result set.

[0151] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the image processing method or the application-side image processing method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the above-mentioned computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0153] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: Acquire a garbage identification result set, wherein the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image; extracting first color data in the first garbage identification frame, and extracting second color data in the second garbage identification frame; Based on the first color data and the second color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated; Based on the color difference ratio, it is determined whether to exclude the first road image and the second road image from the garbage identification result set.

2. The image processing method according to claim 1, characterized in that: Before obtaining the garbage identification result set, the method further includes: Acquire a road image sequence of a target road, wherein the road image sequence includes multiple frames of road images; Inputting the plurality of frames of the road image into a preset garbage recognition model for garbage recognition processing to obtain a garbage recognition frame corresponding to each frame of the road image; The garbage identification result set is constructed based on the road image and the corresponding garbage identification frame.

3. The image processing method according to claim 2, characterized in that: The step of constructing the garbage identification result set based on the road image and the corresponding garbage identification frame includes: Extracting the first road image and the second road image from a plurality of frames of the road image; Expanding the garbage identification frames corresponding to the first road image and the second road image according to a preset ratio to obtain the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image; The garbage identification result set is constructed based on the first garbage identification frame corresponding to the first road image and the second garbage identification frame corresponding to the second road image.

4. The image processing method according to claim 1, wherein: The first color data is composed of a first color code, the second color data is composed of a second color code, and the color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated based on the first color data and the second color data, including: When the first color code and the second color code do not belong to the preset color space, the first color code is subjected to color code conversion processing to obtain first target color data, and the second color code is subjected to color code conversion processing to obtain second target color data; Based on the first target color data and the second target color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame is calculated.

5. The image processing method according to claim 4, characterized in that: The calculating, based on the first target color data and the second target color data, a color difference ratio between the first garbage identification frame and the second garbage identification frame includes: Based on the first target color data and the second target color data, a color difference between each first pixel in the first garbage identification frame and each second pixel in the second garbage identification frame is calculated, and each first pixel corresponds to one second pixel; After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference ratio is obtained.

6. The image processing method according to claim 5, characterized in that: After the color differences between all the first pixels and the corresponding second pixels are calculated, the color difference ratio is obtained, including: After the color differences between all the first pixel points and the corresponding second pixel points are calculated, the color difference between each of the first pixel points and the corresponding second pixel point is obtained; Determine the first pixel point and the second pixel point corresponding to the color difference being greater than a preset color difference threshold as difference pixels; Counting the number of the difference pixels and the total number of the first pixels and the second pixels; The color difference ratio is calculated based on the number of the difference pixels and the total number.

7. The image processing method according to claim 1, characterized in that: The determining, based on the color difference ratio, whether to exclude the first road image and the second road image from the garbage identification result set includes: When the color difference ratio is greater than a preset color difference ratio threshold, removing the first road image and the second road image from the garbage identification result set; When the color difference ratio is less than a preset color difference ratio threshold, the first road image and the second road image are retained in the garbage identification result set.

8. An image processing device, characterized in that: The image processing device comprises: A first acquisition module is used to acquire a garbage identification result set, wherein the garbage identification result set includes a first road image and a first garbage identification frame in the first road image, a second road image and a second garbage identification frame in the second road image; a first extraction module, configured to extract first color data from the first garbage identification frame, and extract second color data from the second garbage identification frame; A first calculation module, configured to calculate a color difference ratio between the first garbage identification frame and the second garbage identification frame based on the first color data and the second color data; The first judgment module is used to judge whether to exclude the first road image and the second road image from the garbage identification result set based on the color difference ratio.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the image processing method according to any one of claims 1 to 7 are implemented.

Images