Smart city monitoring system based on image recognition

Abstract

The invention relates to the technical field of urban road cleaning, in particular to a smart city monitoring system based on image recognition, which comprises a data acquisition module used for acquiring a road monitoring video of an urban environmental sanitation operation area in real time, and a video analysis module used for analyzing a vehicle area and a road area and extracting a road image; the image analysis module is used for analyzing the road cleaning parameters and the road surface dirt parameters; the data storage module is used for storing the road cleaning parameters and analyzing the road cleanliness according to the stored road cleaning parameters; the vehicle acquisition module is used for periodically acquiring operation information during operation of the sanitation vehicle, and the accumulated dust analysis module is used for analyzing the road accumulated dust level according to the operation information; the operation analysis module is used for evaluating the environmental sanitation operation quality; and the cleaning prompting module is used for prompting the sanitation vehicle to perform cleaning operation. According to the invention, accurate supervision of urban road cleaning work is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of urban road cleaning, and in particular to a smart city monitoring system based on image recognition. Background Art

[0002] With the development of the times, existing technologies can rely on the Internet, 5G transmission and modern sensor technology to achieve accurate measurement of the cleaning effect of urban roads. This technology collects data such as road dust accumulation and air environment in real time through on-board equipment, uses intelligent algorithms for comprehensive analysis, and presents the cleanliness of different road sections in a visual way, thereby guiding the optimization of sanitation work, improving the efficiency and quality of urban road cleaning, and promoting the scientific and refined process of urban environmental governance.

[0003] China Patent Publication No.: CN110493572A discloses a smart city monitoring system based on image recognition, including a control terminal for data storage and processing; a garbage sweeper data module with a built-in garbage sweeper working time database; an image acquisition module for collecting road image information and outputting image acquisition signals; an image processing module for processing image information and outputting image clarity signals; an image comparison module for real-time comparison of image clarity at the front and rear ends of the garbage sweeper; an image output module for outputting road re-cleaning signals; a marking module for marking the current road position after cleaning and transmitting it to the control terminal; and a signal transmission module for selecting a garbage sweeper in operation to transmit signals to re-clean the marked road position. This invention compares images before and after the cleaning area of ​​the urban garbage sweeper to achieve management of road cleaning according to road cleanliness, but does not achieve comprehensive analysis of road traffic, road traffic, and sanitation vehicle cleaning data. There are problems of low efficiency in analyzing urban road cleaning data and inaccurate management of urban road cleaning. Summary of the invention

[0004] The object of the present invention is to provide a smart city monitoring system based on image recognition to solve at least one of the problems existing in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A smart city monitoring system based on image recognition, comprising:

[0007] Data acquisition module, used to obtain road monitoring videos of urban sanitation operation areas in real time;

[0008] A video analysis module, used to analyze the vehicle area and the road area according to the road monitoring video, to mark the sanitation vehicle area in the vehicle area, and to extract the road image according to the road monitoring video;

[0009] An image analysis module for analyzing a motion equation based on a road image and a sanitation vehicle area, and for analyzing a road cleanliness parameter and a road surface dirtiness parameter based on the motion equation, the road image and the road area;

[0010] A data storage module, used to store road cleaning parameters and analyze road cleanliness based on the stored road cleaning parameters;

[0011] The vehicle acquisition module is used to periodically obtain the operation information of the sanitation vehicle during operation;

[0012] A dust accumulation analysis module is used to analyze the road dust level based on the operation information;

[0013] Operation analysis module, which is used to evaluate the quality of sanitation operations based on road cleaning parameters, road dirtiness parameters, road cleanliness, operation information and road dust accumulation level;

[0014] The cleaning reminder module is used to remind the sanitation vehicle to perform cleaning operations according to the quality of sanitation operations.

[0015] Further, the video analysis module is provided with a region extraction unit, which is used to extract vehicles in the road monitoring video using vehicle target detection, and to use the region corresponding to the vehicle in the road monitoring video as the vehicle region, and to use the region in the road monitoring video that is not the vehicle region as the road region;

[0016] The video analysis module is also provided with a vehicle marking unit, which is used to mark the vehicle area of ​​the sanitation vehicle in the road monitoring video as a sanitation vehicle area when the sanitation vehicle is located on the road corresponding to the road monitoring video for operation;

[0017] The video analysis module is also provided with a video processing unit, which is used to perform frame extraction processing on the road monitoring video according to a preset frame extraction frequency, extracting k pictures per second from the road monitoring video as road images, wherein k represents the preset frame extraction frequency.

[0018] Furthermore, the image analysis module is provided with a position analysis unit, which is used to analyze the position of the sanitation vehicle according to the image coordinate system and the sanitation vehicle area, and set the sanitation vehicle position to (S1(i), S2(i)), wherein (S1(i), S2(i)) represents the sanitation vehicle position, S1(i) represents the horizontal coordinate of the sanitation vehicle position, which is the mean of the horizontal coordinates of the pixel points in the sanitation vehicle area, S2(i) represents the vertical coordinate of the sanitation vehicle position, which is the mean of the vertical coordinates of the pixel points in the sanitation vehicle area, i represents the image number, i∈N + .

[0019] Furthermore, the image analysis module is also provided with a direction analysis unit, which is used to analyze the movement equation according to the change of the position of the sanitation vehicle. When S1(i)=S1(i-1) and S2(i)=S2(i-1), the position of the sanitation vehicle has not changed, and the direction analysis unit does not analyze the movement equation. When S1(i)≠S1(i-1) or S2(i)≠S2(i-1), the position of the sanitation vehicle has changed, and the direction analysis unit analyzes the movement equation and takes the perpendicular bisector between the current analyzed sanitation vehicle position and the sanitation vehicle position in the previous road image as the movement equation.

[0020] Furthermore, the image analysis module is further provided with a first image analysis unit, which is used to calculate the road cleaning parameter according to the first segmented area, the second segmented area and the road area through a first image analysis formula, and the first image analysis unit is provided with a first image analysis formula as follows:

[0021]

[0022] Wherein, R1(i) represents the road cleaning parameter, H1(i) represents the first grayscale parameter, H2(i) represents the second grayscale parameter, H3(i) represents the third grayscale parameter, L(x, y) represents the grayscale value of each pixel in the road image, M1(i) represents the first segmented area, M2(i) represents the second segmented area, u(i) represents the road area, N1(i) represents the number of pixels in the road area in the first segmented area, and N2(i) represents the number of pixels in the road area in the second segmented area;

[0023] The image analysis module is further provided with a second image analysis unit, which is used to calculate the road surface dirtiness parameter according to the first segmented area, the second segmented area and the road area through a second image analysis formula, and the second image analysis unit is provided with a second image analysis formula as follows:

[0024]

[0025] Among them, R2(i) represents the road surface dirt parameter, K represents the time parameter, and n represents the dirt statistical parameter.

[0026] Furthermore, the data storage module extracts the stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2 as storage comparison parameters, and calculates the road cleanliness through a storage analysis formula according to the storage comparison parameters and the road cleaning parameters. The data storage module is provided with a storage analysis formula as follows:

[0027]

[0028] Where D(i) represents the road cleanliness, r(v) represents the stored road cleaning parameters, v represents the stored image number, and v∈N + , V(i) represents a set of stored image numbers corresponding to stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2, α1 represents a first stored comparison threshold, and α2 represents a second stored comparison threshold.

[0029] Furthermore, the dust analysis module compares the dust concentration with the concentration threshold, and determines the road dust level according to the comparison result. If the comparison result is C(j)≤c1, the dust analysis module determines that the road dust level is level one; if the comparison result is c1<C(j)≤c2, the dust analysis module determines that the road dust level is level two; if the comparison result is C(j)>c2, the dust analysis module determines that the road dust level is level three; wherein C(j) represents the dust concentration, j represents the preset analysis period number, j∈N + , c1 represents the first concentration threshold, and c2 represents the second concentration threshold.

[0030] Further, the operation analysis module is provided with an operation analysis unit, which is used to analyze the quality grade parameter according to the road cleaning parameter and the road neatness, and set the quality grade parameter to F(i), and set F(i)=R1(i)×D(i) / lnH1(i), wherein F(i) represents the quality grade parameter;

[0031] The operation analysis module is also provided with a quality analysis unit, which is used to compare the quality grade parameter with the quality comparison threshold, and evaluate the quality of the sanitation operation according to the comparison result. If the comparison result is F(i)<f1, the quality analysis unit determines that the quality of the sanitation operation is poor; if the comparison result is f1≤F(i)≤f2, the quality analysis unit determines that the quality of the sanitation operation is good; if the comparison result is F(i)>f2, the quality analysis unit determines that the quality of the sanitation operation is excellent; wherein f1 represents the first quality comparison threshold, and f2 represents the second quality comparison threshold.

[0032] Furthermore, the operation analysis module is also provided with a road dust analysis unit, which is used to adjust the analysis process of the quality grade parameters according to the road dust level and the ambient humidity. If the dust level is level one, the road dust analysis unit does not adjust the analysis process of the quality grade parameters; if the dust level is level two and the ambient humidity is less than or equal to the humidity threshold, the road dust analysis unit adjusts the analysis process of the quality grade parameters, and the adjusted quality grade parameters are F1(i); if the dust level is level two and the ambient humidity is greater than the humidity threshold, the road dust analysis unit does not adjust the analysis process of the quality grade parameters; if the dust level is level three, the road dust analysis unit adjusts the analysis process of the quality grade parameters, and the adjusted quality grade parameters are F1(i); wherein, G(j) represents the ambient humidity;

[0033] The operation analysis module is also provided with a road surface dirt analysis unit, which is used to optimize the adjustment process of the quality grade parameter according to the road surface dirt parameter. If the road surface dirt parameter satisfies |R2(i)-1|≤β, the road surface dirt analysis unit does not optimize the adjustment process of the quality grade parameter; if the road surface dirt parameter satisfies |R2(i)-1|>β, the road surface dirt analysis unit optimizes the adjustment process of the quality grade parameter, and the optimized quality grade parameter is F2(i); wherein β represents the dirt comparison threshold.

[0034] Furthermore, the cleaning prompt module prompts the sanitation vehicle to perform cleaning operations according to the quality of the sanitation operations, wherein:

[0035] When the sanitation operation quality of K×k / 2 consecutive image analysis cycles is poor, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned;

[0036] When the quality of sanitation operations for K×k consecutive image analysis cycles is good, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned.

[0037] The beneficial effects of the present invention are as follows: through the acquisition of road monitoring videos and sanitation vehicle operation information by the data acquisition module and the vehicle acquisition module, and the analysis of the acquired data by other modules, a comprehensive analysis of road video data, road cleaning data and road environment data for road cleaning quality evaluation is achieved, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is a schematic diagram of the structure of the smart city monitoring system based on image recognition in this embodiment.

[0040] Figure 2 Schematic diagram of the structure of the video analysis module of this embodiment.

[0041] Figure 3 Schematic diagram of the structure of the image analysis module of this embodiment.

[0042] Figure 4 Schematic diagram of the structure of the job analysis module of this embodiment. DETAILED DESCRIPTION

[0043] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0044] It should be noted that, although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0045] See also Figure 1 As shown, this is a smart city monitoring system based on image recognition in this embodiment, including:

[0046] A data acquisition module is used to acquire in real time road surveillance videos of urban sanitation operation areas, where the urban sanitation operation areas are road areas cleaned by sanitation vehicles in the city, and the road surveillance videos are acquired by importing them through an urban surveillance data platform;

[0047] A video analysis module, used to analyze the vehicle area and the road area according to the road monitoring video, and also used to mark the sanitation vehicle area in the vehicle area, and also used to extract the road image according to the road monitoring video, the video analysis module is connected to the data acquisition module;

[0048] An image analysis module, used to analyze the movement equation according to the road image and the sanitation vehicle area, and also used to analyze the road cleaning parameters and the road surface dirtiness parameters according to the movement equation, the road image and the road area, the image analysis module being connected to the video analysis module;

[0049] A data storage module, used to store road cleaning parameters and analyze road cleanliness according to the stored road cleaning parameters, the data storage module being connected to the image analysis module;

[0050] The vehicle acquisition module is used to periodically acquire the operation information of the sanitation vehicle during operation. The operation information is acquired by uploading it through the sensor installed on the sanitation vehicle. The operation information includes dust concentration and ambient humidity. The dust concentration is the PM10 particle concentration on the road when the sanitation vehicle is operating, and its unit is μg / m 3 In this embodiment, the acquisition frequency of the operation information is once per preset analysis period. In this embodiment, the preset analysis period is set to 10 seconds. In this embodiment, the setting of the preset analysis period is not specifically limited. Those skilled in the art can freely set it. For example, it can also be set to 5 seconds, 15 seconds, and 20 seconds. The setting of the preset analysis period should meet the requirement of being less than or equal to 30 seconds.

[0051] A dust accumulation analysis module, used to analyze the road dust level according to the operation information, and the dust accumulation analysis module is connected to the vehicle acquisition module;

[0052] An operation analysis module, used to evaluate the quality of sanitation operations according to road cleaning parameters, road dirtiness parameters, road cleanliness, operation information and road dust accumulation level, the operation analysis module being connected to the data storage module and the dust accumulation analysis module;

[0053] The cleaning prompt module is used to prompt the sanitation vehicle to perform cleaning operations according to the quality of the sanitation operations. The cleaning prompt module is connected to the operation analysis module.

[0054] Specifically, this embodiment is applied to the urban monitoring data processing cloud, and the cleaning quality evaluation of the road cleaning operation is achieved by analyzing the road monitoring video and the environmental data of the sanitation vehicle during the cleaning operation.

[0055] See also Figure 2 As shown, the video analysis module includes:

[0056] A region extraction unit, used to extract a vehicle region and a road region according to a road monitoring video;

[0057] A vehicle marking unit, used to mark the sanitation vehicle area according to the vehicle area, and the vehicle marking unit is connected to the area extraction unit;

[0058] The video processing unit is used to extract the road image according to the road monitoring video, and the video processing unit is connected to the vehicle marking unit.

[0059] Specifically, the region extraction unit in this embodiment uses vehicle target detection to extract vehicles in the road monitoring video, uses the region corresponding to the vehicle in the road monitoring video as the vehicle region, and uses the region in the road monitoring video that is not the vehicle region as the road region.

[0060] Specifically, the region extraction unit in this embodiment uses YOLOv5-based vehicle target detection to process the road surveillance video. The YOLOv5-based vehicle target detection is a deep learning target detection algorithm. The YOLOv5 algorithm is an open source target detection algorithm that can be run using Python. Vehicle detection data sets such as UA-DETRAC vehicle detection data set, Vehicle-Dataset vehicle detection data set and BITVeh icle vehicle detection data set are used to train its model to achieve detection of vehicle targets.

[0061] Specifically, in this embodiment, the area extraction unit analyzes the road monitoring video to extract the vehicle area and road area in the road monitoring video, thereby realizing the extraction of the vehicle position in the video, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0062] Specifically, when the sanitation vehicle is operating on the road corresponding to the road monitoring video, the vehicle marking unit in this embodiment marks the vehicle area of ​​the sanitation vehicle in the road monitoring video as the sanitation vehicle area.

[0063] It can be understood that the method of marking the sanitation vehicle area is not specifically limited in this embodiment, and technical personnel in this field can freely set it, such as using the sanitation vehicle model to train the vehicle target detection model to extract the area corresponding to the sanitation vehicle, and can also use manual labeling methods to label the sanitation vehicle.

[0064] Specifically, the video processing unit in this embodiment performs frame extraction processing on the road monitoring video according to the preset frame extraction frequency, and extracts k pictures per second from the road monitoring video as the road image, wherein k represents the preset frame extraction frequency, and 1≤k≤5. The preset frame extraction frequency represents the number of pictures extracted from the video per second. It can be understood that the setting of the preset frame extraction frequency is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it meets the analysis of the road image.

[0065] It can be understood that the frame extraction process is not specifically limited in this embodiment, and those skilled in the art can freely set it. For example, the video can be subjected to frame extraction processing through the ffmpeg -i nputfile.avi -r 1 -fimage2 image-%05d.jpeg command, where -r represents the frame extraction frequency and -f represents the picture saving format.

[0066] Specifically, in this embodiment, the video processing unit performs frame extraction processing on the road monitoring video to obtain a road image, thereby extracting pictures for road cleaning analysis from the road monitoring video, increasing the diversity of sample analysis, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0067] See also Figure 3 As shown, the image analysis module includes:

[0068] A coordinate construction unit, used to construct an image coordinate system according to the road image;

[0069] A position analysis unit, used to analyze the position of the sanitation vehicle according to the image coordinate system and the sanitation vehicle area, the position analysis unit being connected to the coordinate construction unit;

[0070] A direction analysis unit, used to analyze the movement equation according to the position of the sanitation vehicle, the direction analysis unit being connected to the position analysis unit;

[0071] A first image analysis unit, used for analyzing the road cleaning parameters according to the movement equation, the image coordinate system and the road area, the first image analysis unit being connected to the direction analysis unit;

[0072] The second image analysis unit is used to analyze the road surface dirt parameters according to the movement equation, the image coordinate system and the road area. The second image analysis unit is connected to the direction analysis unit.

[0073] Specifically, the coordinate construction unit described in this embodiment uses the pixel point at the lower left corner of the road image as the coordinate origin, and uses the two sides adjacent to the coordinate origin as the x-axis and y-axis respectively. The x-axis increases from left to right, and the y-axis increases from bottom to top. The unit length is 1 pixel. A plane rectangular coordinate system is established as the image coordinate system, and the position of each pixel in the road image is represented by rectangular coordinates.

[0074] Specifically, the position analysis unit in this embodiment analyzes the position of the sanitation vehicle according to the image coordinate system and the sanitation vehicle area, and sets the sanitation vehicle position to (S1(i), S2(i)), where (S1(i), S2(i)) represents the sanitation vehicle position, and S1(i) represents the horizontal coordinate of the sanitation vehicle position, which is the mean of the horizontal coordinates of the pixel points in the sanitation vehicle area, recorded as S2(i) represents the ordinate of the sanitation vehicle position, which is the mean of the ordinates of the pixels in the sanitation vehicle area, denoted by i represents the image number, i∈N + , (x, y) represents the coordinate point in the image coordinate system, U(i) represents the sanitation vehicle area, and NS(i) represents the number of pixels in the sanitation vehicle area.

[0075] Specifically, in this implementation, the position analysis unit analyzes the image coordinate system and the sanitation vehicle area to analyze the sanitation vehicle position, and the sanitation vehicle position is used to represent the position data of the sanitation vehicle area in the road image, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0076] Specifically, the direction analysis unit in this embodiment analyzes the movement equation according to the change of the position of the sanitation vehicle. When S1(i)=S1(i-1) and S2(i)=S2(i-1), the position of the sanitation vehicle has not changed, and the direction analysis unit does not analyze the movement equation. When S1(i)≠S1(i-1) or S2(i)≠S2(i-1), the position of the sanitation vehicle has changed, and the direction analysis unit analyzes the movement equation, takes the perpendicular bisector between the current analysis position of the sanitation vehicle and the position of the sanitation vehicle in the previous road image as the movement equation, and sets the movement equation as follows:

[0077]

[0078] Among them, W1(i) represents the horizontal coordinate parameter of the movement equation, W2(i) represents the horizontal coordinate parameter of the movement equation, and (S2(i-1), S2(i-1)) represents the position of the sanitation vehicle in the previous road image.

[0079] Specifically, in this implementation, the direction analysis unit analyzes the position of the sanitation vehicle to derive a movement equation, and uses the movement equation to represent the cleaning boundary line in the direction of travel of the sanitation vehicle, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0080] Specifically, in this embodiment, the first image analysis unit divides the road image into two parts, the dividing line is the straight line corresponding to the moving equation in the image coordinate system, and the part where the sanitation vehicle is located in the currently analyzed road image is taken as the first segmentation area, and the part where the sanitation vehicle is located in the previous road image is taken as the second segmentation area.

[0081] Specifically, in this embodiment, the first image analysis unit calculates the road cleaning parameter according to the first segmented area, the second segmented area and the road area through a first image analysis formula, and the first image analysis unit is provided with a first image analysis formula as follows:

[0082]

[0083]

[0084] Among them, R1(i) represents the road cleaning parameter, H1(i) represents the first grayscale parameter, H2(i) represents the second grayscale parameter, H3(i) represents the third grayscale parameter, L(x, y) represents the grayscale value of each pixel in the road image, M1(i) represents the first segmentation area, M2(i) represents the second segmentation area, u(i) represents the road area, N1(i) represents the number of pixels in the road area within the first segmentation area, and N2(i) represents the number of pixels in the road area within the second segmentation area.

[0085] Specifically, in this embodiment, the first image analysis unit analyzes the movement equation, image coordinate system and road area to analyze the road cleaning parameters, and uses the road cleaning parameters to represent the change characteristics between road images after cleaning by sanitation vehicles, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0086] Specifically, in this embodiment, the second image analysis unit calculates the road surface dirt parameter according to the first segmented area, the second segmented area and the road area through a second image analysis formula, and the second image analysis unit is provided with a second image analysis formula as follows:

[0087]

[0088] Wherein, R2(i) represents the road surface dirt parameter, K represents the time parameter, 15≤K≤30, and n represents the dirt statistical parameter, n∈N and n≤K×k. It can be understood that the value of the time parameter is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it satisfies the analysis of the road surface dirt parameter. The optimal value of the time parameter is: K=30.

[0089] Specifically, in this embodiment, the second image analysis unit analyzes the movement equation, image coordinate system and road area to analyze the road surface dirtiness parameters, and uses the road surface dirtiness parameters to represent the changing characteristics of the road image in each time period after cleaning by the sanitation vehicle, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0090] Specifically, the data storage module in this embodiment extracts the stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2 as storage comparison parameters, and calculates the road cleanliness through a storage analysis formula based on the storage comparison parameters and the road cleaning parameters. The data storage module is provided with a storage analysis formula as follows:

[0091]

[0092] Where D(i) represents the road cleanliness, r(v) represents the stored road cleaning parameters, v represents the stored image number, and v∈N + , V(i) represents the set of stored image numbers corresponding to the stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2, α1 represents the first storage comparison threshold, 0.7≤α1≤0.9, α2 represents the second storage comparison threshold, 1.1≤α2≤1.3. It can be understood that the value of the storage comparison threshold is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it satisfies the analysis of road cleanliness. The optimal value of the storage comparison threshold is: α1=0.8, α2=1.2.

[0093] Specifically, in this embodiment, the data storage module analyzes the stored road cleaning data to extract data in the historical data that is close in order of magnitude to the current data for comparison, thereby analyzing the road cleanliness, and using the road cleanliness to represent the characteristic relationship between the road cleaning features in the current image and the historical data, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0094] Specifically, the dust analysis module in this embodiment compares the dust concentration with the concentration threshold, and determines the road dust level according to the comparison result. If the comparison result is C(j)≤c1, the dust analysis module determines that the road dust level is level one; if the comparison result is c1<C(j)≤c2, the dust analysis module determines that the road dust level is level two; if the comparison result is C(j)>c2, the dust analysis module determines that the road dust level is level three; wherein C(j) represents the dust concentration, j represents the preset analysis period number, j∈N + , c1 represents the first concentration threshold, 50≤c1≤100, c2 represents the second concentration threshold, 200≤c2≤350. It can be understood that the concentration threshold is not specifically limited in this embodiment, and can be freely set by those skilled in the art, as long as it meets the analysis of the road dust level. The optimal values ​​of the concentration threshold are: c1=50, c2=250.

[0095] Specifically, in this embodiment, the dust analysis module analyzes the dust concentration to analyze the road dust level and implements the level evaluation of the dust amount on the road, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0096] See also Figure 4 As shown, the job analysis module includes:

[0097] An operation analysis unit for analyzing quality grade parameters based on road cleanliness parameters and road neatness;

[0098] A road dust analysis unit, used to adjust the analysis process of the quality grade parameter according to the road dust grade and the operation information, the road dust analysis unit being connected to the operation analysis unit;

[0099] A road contamination analysis unit, used for optimizing the adjustment process of the quality grade parameter according to the road contamination parameter, the road contamination analysis unit being connected to the road dust analysis unit;

[0100] The quality analysis unit is used to evaluate the quality of sanitation work according to the quality grade parameters, and the quality analysis unit is connected to the road surface dirt analysis unit.

[0101] Specifically, the operation analysis unit described in this embodiment analyzes the quality grade parameter according to the road cleaning parameter and the road neatness, sets the quality grade parameter to F(i), and sets F(i)=R1(i)×D(i) / l nH1(i), where F(i) represents the quality grade parameter.

[0102] Specifically, in this embodiment, the operation analysis unit analyzes the road cleaning parameters and road cleanliness to analyze the quality grade parameters, and uses the quality grade parameters to represent the road cleaning quality characteristics of the sanitation vehicle cleaning operation, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0103] Specifically, the road dust analysis unit in this embodiment adjusts the analysis process of the quality level parameter according to the road dust level and the ambient humidity. If the dust level is level one, the road dust analysis unit does not adjust the analysis process of the quality level parameter; if the dust level is level two and the ambient humidity is less than or equal to the humidity threshold, the road dust analysis unit adjusts the analysis process of the quality level parameter, and the adjusted quality level parameter is F1(i), and F1(i)=F(i)×[ln(e+G(j))] is set. 2 ; If the dust accumulation level is level 2 and the ambient humidity is greater than the humidity threshold, the road dust accumulation analysis unit does not adjust the analysis process of the quality level parameter; if the dust accumulation level is level 3, the road dust accumulation analysis unit adjusts the analysis process of the quality level parameter, and the adjusted quality level parameter is F1(i), and F1(i)=F(i)×[ln(e+G(j))] 3; Wherein, G(j) represents the ambient humidity, and the humidity threshold is set to g, 0.3≤g≤0.5. It can be understood that the humidity threshold is not specifically limited in this embodiment, and can be freely set by those skilled in the art, as long as the adjustment of the quality grade parameter analysis process is met. The optimal value of the humidity threshold is: g=0.4.

[0104] Specifically, in this embodiment, the road dust analysis unit analyzes the road dust level and operation information to adjust the analysis process of the quality grade parameters, so that the adjusted quality grade parameters are related to the dust data and environmental data on the road, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0105] Specifically, the road contamination analysis unit in this embodiment optimizes the adjustment process of the quality grade parameter according to the road contamination parameter. If the road contamination parameter satisfies |R2(i)-1|≤β, the road contamination analysis unit does not optimize the adjustment process of the quality grade parameter. If the road contamination parameter satisfies |R2(i)-1|>β, the road contamination analysis unit optimizes the adjustment process of the quality grade parameter. The optimized quality grade parameter is F2(i), and F2(i)=F1(i)×e -|R2(i)-1| ; Wherein, β represents the dirt comparison threshold, 0.1≤β≤0.3. It can be understood that the value of the dirt comparison threshold is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as the optimization of the quality level parameter adjustment process is satisfied. The optimal value of the dirt comparison threshold is: β=0.2.

[0106] Specifically, in this embodiment, the road surface dirtiness analysis unit analyzes the road surface dirtiness parameters to optimize the adjustment process of the quality grade parameters, so that the optimized quality grade parameters are correlated with the changes in the road image in each time period after cleaning by the sanitation vehicle, thereby improving the system's analysis efficiency of urban road cleaning data and improving the accuracy of urban road cleaning management.

[0107] Specifically, the quality analysis unit described in this embodiment compares the quality grade parameter with the quality comparison threshold, and evaluates the quality of the sanitation operation according to the comparison result. If the comparison result is F(i)<f1, the quality analysis unit determines that the quality of the sanitation operation is poor; if the comparison result is f1≤F(i)≤f2, the quality analysis unit determines that the quality of the sanitation operation is good; if the comparison result is F(i)>f2, the quality analysis unit determines that the quality of the sanitation operation is excellent; wherein, f1 represents the first quality comparison threshold, 1.2≤f1≤1.4, and f2 represents the second quality comparison threshold, 1.7≤f2≤1.9. It can be understood that the value of the quality comparison threshold is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it satisfies the analysis of the quality of the sanitation operation. The optimal value of the quality comparison threshold is: f1=1.3, f2=1.8.

[0108] Specifically, the cleaning prompt module in this embodiment manages the sanitation vehicle to perform cleaning operations according to the quality of sanitation operations, wherein:

[0109] When the sanitation operation quality of K×k / 2 consecutive image analysis cycles is poor, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned;

[0110] When the quality of sanitation operations for K×k consecutive image analysis cycles is good, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned.

[0111] Specifically, the cleaning prompt module in this embodiment prompts the sanitation vehicle to perform cleaning operations only in the above two situations, and does not prompt the sanitation vehicle to perform cleaning operations in other situations.

[0112] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A smart city monitoring system based on image recognition, characterized in that: include: Data acquisition module, used to obtain road monitoring videos of urban sanitation operation areas in real time; A video analysis module, used to analyze the vehicle area and the road area according to the road monitoring video, to mark the sanitation vehicle area in the vehicle area, and to extract the road image according to the road monitoring video; An image analysis module for analyzing a motion equation based on a road image and a sanitation vehicle area, and for analyzing a road cleanliness parameter and a road surface dirtiness parameter based on the motion equation, the road image and the road area; A data storage module, used to store road cleaning parameters and analyze road cleanliness based on the stored road cleaning parameters; The vehicle acquisition module is used to periodically obtain the operation information of the sanitation vehicle during operation; A dust accumulation analysis module is used to analyze the road dust level based on the operation information; Operation analysis module, which is used to evaluate the quality of sanitation operations based on road cleaning parameters, road dirtiness parameters, road cleanliness, operation information and road dust accumulation level; The cleaning reminder module is used to remind the sanitation vehicle to perform cleaning operations according to the quality of sanitation operations.

2. The smart city monitoring system based on image recognition according to claim 1 is characterized in that: The video analysis module is provided with a region extraction unit, which is used to extract vehicles in the road monitoring video using vehicle target detection, and use the region corresponding to the vehicle in the road monitoring video as the vehicle region, and use the region in the road monitoring video that is not the vehicle region as the road region; The video analysis module is also provided with a vehicle marking unit, which is used to mark the vehicle area of ​​the sanitation vehicle in the road monitoring video as a sanitation vehicle area when the sanitation vehicle is located on the road corresponding to the road monitoring video for operation; The video analysis module is also provided with a video processing unit, which is used to perform frame extraction processing on the road monitoring video according to a preset frame extraction frequency, extracting k pictures per second from the road monitoring video as road images, wherein k represents the preset frame extraction frequency.

3. The smart city monitoring system based on image recognition according to claim 2 is characterized in that: The image analysis module is provided with a position analysis unit, which is used to analyze the position of the sanitation vehicle according to the image coordinate system and the sanitation vehicle area, and set the sanitation vehicle position to (S1(i), S2(i)), wherein (S1(i), S2(i)) represents the sanitation vehicle position, S1(i) represents the horizontal coordinate of the sanitation vehicle position, which is the mean of the horizontal coordinates of the pixel points in the sanitation vehicle area, S2(i) represents the vertical coordinate of the sanitation vehicle position, which is the mean of the vertical coordinates of the pixel points in the sanitation vehicle area, i represents the image number, i∈N + .

4. The smart city monitoring system based on image recognition according to claim 3 is characterized in that: The image analysis module is also provided with a direction analysis unit, which is used to analyze the movement equation according to the change of the position of the sanitation vehicle. When S1(i)=S1(i-1) and S2(i)=S2(i-1), the position of the sanitation vehicle has not changed, and the direction analysis unit does not analyze the movement equation. When S1(i)≠S1(i-1) or S2(i)≠S2(i-1), the position of the sanitation vehicle has changed, and the direction analysis unit analyzes the movement equation and takes the perpendicular bisector between the current analyzed sanitation vehicle position and the sanitation vehicle position in the previous road image as the movement equation.

5. The smart city monitoring system based on image recognition according to claim 4 is characterized in that: The image analysis module is further provided with a first image analysis unit, which is used to calculate the road cleaning parameter according to the first segmented area, the second segmented area and the road area through a first image analysis formula, and the first image analysis unit is provided with a first image analysis formula as follows: Wherein, R1(i) represents the road cleaning parameter, H1(i) represents the first grayscale parameter, H2(i) represents the second grayscale parameter, H3(i) represents the third grayscale parameter, L(x, y) represents the grayscale value of each pixel in the road image, M1(i) represents the first segmented area, M2(i) represents the second segmented area, u(i) represents the road area, N1(i) represents the number of pixels in the road area in the first segmented area, and N2(i) represents the number of pixels in the road area in the second segmented area; The image analysis module is further provided with a second image analysis unit, which is used to calculate the road surface dirtiness parameter according to the first segmented area, the second segmented area and the road area through a second image analysis formula, and the second image analysis unit is provided with a second image analysis formula as follows: Among them, R2(i) represents the road surface dirt parameter, K represents the time parameter, and n represents the dirt statistical parameter.

6. The smart city monitoring system based on image recognition according to claim 5 is characterized in that: The data storage module extracts the stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2 as storage comparison parameters, and calculates the road cleanliness through a storage analysis formula according to the storage comparison parameters and the road cleaning parameters. The data storage module is provided with a storage analysis formula as follows: Where D(i) represents the road cleanliness, r(v) represents the stored road cleaning parameters, v represents the stored image number, and v∈N + , V(i) represents a set of stored image numbers corresponding to stored road cleaning parameters satisfying α1≤R1(i) / r(v)≤α2, α1 represents a first stored comparison threshold, and α2 represents a second stored comparison threshold.

7. The smart city monitoring system based on image recognition according to claim 1 is characterized in that: The dust analysis module compares the dust concentration with the concentration threshold, and determines the road dust level according to the comparison result. If the comparison result is C(j)≤c1, the dust analysis module determines that the road dust level is level one; if the comparison result is c1<C(j)≤c2, the dust analysis module determines that the road dust level is level two; if the comparison result is C(j)>c2, the dust analysis module determines that the road dust level is level three; wherein C(j) represents the dust concentration, j represents the preset analysis period number, j∈N + , c1 represents the first concentration threshold, and c2 represents the second concentration threshold.

8. The smart city monitoring system based on image recognition according to claim 6 is characterized in that: The operation analysis module is provided with an operation analysis unit, which is used to analyze the quality grade parameter according to the road cleaning parameter and the road neatness, and set the quality grade parameter to F(i), and set F(i)=R1(i)×D(i) / lnH1(i), wherein F(i) represents the quality grade parameter; The operation analysis module is also provided with a quality analysis unit, which is used to compare the quality grade parameter with the quality comparison threshold, and evaluate the quality of the sanitation operation according to the comparison result. If the comparison result is F(i)<f1, the quality analysis unit determines that the quality of the sanitation operation is poor; if the comparison result is f1≤F(i)≤f2, the quality analysis unit determines that the quality of the sanitation operation is good; if the comparison result is F(i)>f2, the quality analysis unit determines that the quality of the sanitation operation is excellent; wherein f1 represents the first quality comparison threshold, and f2 represents the second quality comparison threshold.

9. The smart city monitoring system based on image recognition according to claim 8 is characterized in that: The operation analysis module is further provided with a road dust analysis unit, which is used to adjust the analysis process of the quality grade parameter according to the road dust level and the ambient humidity. If the dust level is level one, the road dust analysis unit does not adjust the analysis process of the quality grade parameter; if the dust level is level two and the ambient humidity is less than or equal to the humidity threshold, the road dust analysis unit adjusts the analysis process of the quality grade parameter, and the adjusted quality grade parameter is F1(i); if the dust level is level two and the ambient humidity is greater than the humidity threshold, the road dust analysis unit does not adjust the analysis process of the quality grade parameter; if the dust level is level three, the road dust analysis unit adjusts the analysis process of the quality grade parameter, and the adjusted quality grade parameter is F1(i); wherein G(j) represents the ambient humidity; The operation analysis module is also provided with a road surface dirt analysis unit, which is used to optimize the adjustment process of the quality grade parameter according to the road surface dirt parameter. If the road surface dirt parameter satisfies |R2(i)-1|≤β, the road surface dirt analysis unit does not optimize the adjustment process of the quality grade parameter; if the road surface dirt parameter satisfies |R2(i)-1|>β, the road surface dirt analysis unit optimizes the adjustment process of the quality grade parameter, and the optimized quality grade parameter is F2(i); wherein β represents the dirt comparison threshold.

10. The smart city monitoring system based on image recognition according to claim 9 is characterized in that: The cleaning prompt module prompts the sanitation vehicle to perform cleaning operations according to the quality of the sanitation operations, wherein: When the sanitation operation quality of K×k / 2 consecutive image analysis cycles is poor, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned; When the quality of sanitation operations for K×k consecutive image analysis cycles is good, the cleaning prompt module prompts the sanitation vehicle that the road corresponding to the current road monitoring video needs to be cleaned.

Images