Shared bicycle specification degree evaluation method under view angle of unmanned aerial vehicle

Abstract

The invention relates to the field of intelligent city management and unmanned aerial vehicle application, and provides a shared bicycle specification degree evaluation method under the view angle of an unmanned aerial vehicle. According to the method, a shared bicycle parking area is inspected through an unmanned aerial vehicle, a high-definition camera carried by the unmanned aerial vehicle is used for collecting image data in real time, and the parking state of the shared bicycle is comprehensively evaluated through an image processing and analysis technology. The evaluation process comprises three core steps: lodging detection, uniformity evaluation and mixing degree evaluation. According to the invention, by introducing the unmanned aerial vehicle technology, the automatic inspection of the shared bicycle parking area and the evaluation of the shared bicycle parking specification degree are realized, the evaluation efficiency is significantly improved, powerful technical support is provided for urban shared bicycle management, and the improvement of the urban image and the optimization of the outgoing experience of citizens are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of urban integrated management, and in particular to a method for evaluating the standardization of shared bicycles from the perspective of an unmanned aerial vehicle. Background Art

[0002] The convenience and environmental friendliness of shared bicycles have played an important role in urban travel, but the problem of disorderly parking has also become a major problem for urban management. This disorderly parking not only damages the image of the city, but also brings many negative effects: (1) From the perspective of urban environment and aesthetics, the disorderly parking of shared bicycles has seriously affected the cleanliness and aesthetics of the city. A large number of disorderly parked shared bicycles occupy public spaces such as sidewalks and green belts, making the originally spacious pedestrian areas crowded. This not only reduces the travel experience of citizens, but also damages the environmental sanitation and landscape image of the city. (2) From the perspective of traffic order and safety, the disorderly parking of shared bicycles has brought great hidden dangers to urban traffic. Randomly parked shared bicycles may block traffic arteries, hinder the normal travel of motor vehicles and non-motor vehicles, and increase the risk of traffic accidents. Especially at busy traffic intersections and narrow streets, disorderly parked shared bicycles may become a "stumbling block" to traffic, posing a serious threat to the travel safety of citizens. (3) From the perspective of urban management, the disorderly parking of shared bicycles has brought great challenges to urban management. Due to the large number of shared bicycles and their high mobility, traditional urban management methods are difficult to deal with effectively. At the same time, some users deliberately park their bicycles in non-designated areas, which increases the difficulty and cost of urban management.

[0003] The basis and premise for the maintenance and management of shared bicycles is to detect and evaluate the parking status of shared bicycles. In the prior art, the parking status of shared bicycles is mainly detected by manual inspection, that is, by sending special personnel to inspect. However, this method has problems such as low inspection efficiency and high labor costs. Manual inspections are difficult to fully cover all areas, and are prone to omissions and blind spots. At the same time, due to limited manpower, it is difficult to cope with the problem of large numbers and high mobility of shared bicycles.

[0004] Therefore, how to obtain the parking status of shared bicycles in a low-cost, efficient and accurate manner is a key and difficult problem that needs to be solved at present. Summary of the invention

[0005] In view of the defects of the prior art, the present invention provides a method for evaluating the standardization of shared bicycles from the perspective of a drone, which can effectively solve the above problems.

[0006] The technical solution adopted by the present invention is as follows:

[0007] The present invention provides a method for evaluating the standardization of shared bicycles from the perspective of an unmanned aerial vehicle, comprising the following steps:

[0008] Step S1: Image acquisition:

[0009] The drone is equipped with a high-definition camera and conducts inspections along the specified route. During the inspection process, it collects images of the shared bicycle parking area in real time.

[0010] Step S2: Lodging detection:

[0011] Use image recognition technology to process and analyze the collected images of shared bicycle parking areas to identify fallen shared bicycles;

[0012] Step S3: Regularity evaluation:

[0013] Based on the detection of falling over, the neatness of the shared bicycle parking area is further evaluated: first, the outline of each shared bicycle in the shared bicycle parking area image is extracted through image processing technology, and the center of mass position of each shared bicycle is calculated; then, a point cloud is drawn on the blank image according to the center of mass position of the shared bicycle, and a linear fit is performed, and the neatness is evaluated according to the angle between the parking edge and the fitted straight line;

[0014] Step S4: Mixing degree evaluation:

[0015] Evaluate whether there is a mixed placement phenomenon in the shared bicycle parking area image, that is, whether shared bicycles of different types or brands are mixed together: Use color recognition and image segmentation technology to classify shared bicycles in the shared bicycle parking area image by color, and calculate the spatial distribution entropy of each type of shared bicycle in the shared bicycle parking area; Based on the spatial distribution entropy and the area proportion of the mixed placement area, evaluate the mixed placement degree in the shared bicycle parking area;

[0016] Step S5: Health score:

[0017] The health of the shared bicycle parking area is scored based on the results of lodging detection, neatness assessment, and mixed degree assessment. The method is as follows: the scoring criteria and weights of different indicators are set, and according to the scores of each indicator, the final health score H is obtained using the following formula:

[0018]

[0019] Where: δ1, δ2, δ3 are the proportions of the evaluation factors of falling, neatness and mixed degree, and δ1+δ2+δ3=1; y1 is the number of fallen bicycles, Y is the total number of shared bicycles, N is the neatness score, and M is the mixed degree score.

[0020] Preferably, step S1 specifically comprises:

[0021] The drone is a multi-rotor drone with vertical take-off and landing and hovering functions. It is equipped with a gimbal camera, a high-precision positioning module, and an obstacle avoidance sensor. The gimbal camera has wide-angle, zoom, and infrared shooting functions.

[0022] Preferably, in step S2, a fall detection module is used to detect the fall of shared bicycles, which uses deep learning technology to automatically learn and identify shared bicycles and their fall status by training a large amount of sample data, and accurately locates the position of the fallen shared bicycle and the time when the fall occurred in combination with GPS information and timestamps; specifically, it includes two stages: shared bicycle area interception and fall identification, including the following steps:

[0023] Step S2.1, constructing a training sample set of fallen shared bicycles; the training sample set includes multiple training samples; each training sample is a picture of a fallen shared bicycle; the pictures of fallen shared bicycles include pictures taken from a drone perspective and open source pictures; the pictures of fallen shared bicycles have a minimum circumscribed rectangle of the fallen shared bicycle, forming a labeling box of the fallen shared bicycle;

[0024] The shared bicycle lying down picture is a shared bicycle lying down picture whose image quality is higher than a set value after being evaluated by an image quality evaluation function;

[0025] The image quality evaluation function is:

[0026]

[0027] Where: α1, α2, α3 are the proportions of subjective evaluation factors, objective evaluation factors and image peak signal-to-noise ratio, α1<α2<α3, and α1+α2+α3=1; F i is the score given to the image through subjective evaluation for the i-th time, n is the number of subjective evaluations; Z j is the score given to the image through objective evaluation for the jth time, m is the number of objective evaluations; χ represents the peak signal-to-noise ratio of the image;

[0028] Step S2.2, introducing the shared bicycle collapse training sample set into the feature pyramid structure deep learning network of the residual attention mechanism for training, and obtaining a trained collapse detection model;

[0029] Step S2.2.1, pre-processing each shared bicycle lying training sample input into the network and adjusting it to a specified size;

[0030] Step S2.2.2, split the preprocessed shared bicycle lying training samples into SxS cells, each cell is used to detect the shared bicycle image whose center point falls on the cell, and each cell obtains the shared bicycle prediction box and its confidence;

[0031] Among them: the confidence of the shared bicycle lying information is calculated by accuracy, precision and recall, all of which are calculated through confusion matrix;

[0032] Step S2.2.3, each cell gives the probability value of its predicted category, denoted as Pr;

[0033] Step S2.2.4, traverse all cells and filter the set of cell center points whose intersection-over-union ratio is greater than the threshold, that is: if the intersection-over-union ratio is greater than the threshold, the cell center point is retained, otherwise, the cell center point is discarded, assuming that a total of w grid center points are obtained;

[0034] In step S2.2.5, the accuracy of the bounding box is expressed by the intersection-over-union ratio between the shared bike prediction box and the actual box, denoted as IOU;

[0035] Step S2.2.6, for the w grid center points selected in step S2.2.4, use the loss function to calculate the loss value. If the loss value is less than the set value, the training of the deep learning network is completed, and a trained lodging detection model is obtained, and step S2.3 is executed; otherwise, the parameters of the deep learning network are adjusted according to the loss value to obtain an updated deep learning network. For the updated deep learning network, the remaining shared bicycle lodging training samples are used to continue training; this is continuously iterated until a trained deep learning network is obtained, and step S2.3 is executed;

[0036] Step S2.3, using the trained fall detection model, detect and capture the input shared bicycle parking area image.

[0037] Preferably, in step S3, the neatness assessment is as follows: the neatness of shared bicycle parking is analyzed by calculating the angle between a line connecting the centers of mass of each shared bicycle and a reference line.

[0038] Preferably, step S3 specifically includes: using a neatness assessment module to accurately assess the neatness of shared bicycle parking, and the neatness assessment module combines GPS information and timestamps to accurately track the location of each shared bicycle parking area and accurately record the inspection time, specifically including:

[0039] Step S3.1: Color space conversion and color recognition:

[0040] Convert the shared bicycle parking area image from RGB space to HSV space; in HSV space, identify the color of shared bicycles by setting the threshold range and construct the corresponding color mask image;

[0041] Step S3.2: Shared bicycle contour extraction and centroid positioning:

[0042] Using image processing technology, the shared bicycle outlines are extracted from the color mask image; for each shared bicycle outline, its centroid position is calculated as the representative point of the shared bicycle in the shared bicycle parking area image;

[0043] Step S3.3: Point cloud generation and linear fitting:

[0044] On the blank image, according to the color and centroid position of the shared bicycle, draw points of the same color at the corresponding centroid position to form a point cloud map;

[0045] According to the distribution of the point cloud in the point cloud map, it is divided into two areas, the upper and lower areas; linear fitting is performed on the point clouds of the overall area and the upper and lower areas respectively, and three fitting lines are obtained, namely: the overall fitting line, the upper half fitting line and the lower half fitting line;

[0046] Step S3.4: Road edge recognition and angle measurement:

[0047] Using image processing technology, the road boundary lines in the shared bicycle parking area image are identified and their directions are determined;

[0048] The angles between the overall fitting straight line, the upper half fitting straight line, the lower half fitting straight line and the road boundary line are measured respectively;

[0049] Step S3.5: Neatness score calculation:

[0050] Determine the importance of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle in evaluating the neatness of shared bicycles, and assign corresponding weights, β1, β2, and β3, respectively:

[0051] The neatness score N is obtained according to the following formula:

[0052] N=x1×β1+x2×β2+x3×β3

[0053] Where x1, x2, and x3 are the scores of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle, respectively.

[0054] Preferably, in step S4, a mixed degree assessment module is used to assess whether there is a mixed parking situation of shared bicycles of different types or brands in the shared bicycle parking area image. At the same time, combined with GPS information and timestamp, the location of each shared bicycle parking area is accurately tracked through GPS information, and the accurate time of inspection is recorded through timestamp. The specific method is:

[0055] Step S4.1: Point cloud acquisition and mixed area identification:

[0056] Obtain the point cloud image generated according to the color of the shared bicycles in step S3.3; in the point cloud image, determine whether there is a mixed phenomenon of shared bicycles based on the color information; if so, extract the point cloud data of the mixed area and execute step S4.2;

[0057] Step S4.2: Spatial distribution entropy evaluation:

[0058] The mixed area is divided into multiple grids, and the spatial distribution entropy of each grid is calculated using the following formula;

[0059]

[0060] Among them, E u is the spatial distribution entropy of the u-th grid, V is the number of colors of shared bicycles, and p uv is the ratio of the number of shared bicycles of the vth color in the uth grid to the total number of shared bicycles in the grid;

[0061] The spatial distribution entropy of each grid in the mixed area is averaged to obtain the average spatial distribution entropy of the whole mixed area E. avg :

[0062] Step S4.3: Quantification and degree assessment of mixed areas:

[0063] Determine the ratio of the mixed area to the total area of ​​shared bicycles, expressed as the mixed area area ratio A:

[0064] Combined with the area ratio A of the mixed area and the average value E of the spatial distribution entropy of the mixed area as a whole avg , use the following formula to get the mixing degree score M:

[0065] M=ω A ×A+ω E ×E avg

[0066] Where: A is the weight of the area ratio of the mixed area, A is the area ratio of the mixed area; ω E is the weight of the spatial distribution entropy of the entire mixing area, E avg It is the average value of the spatial distribution entropy of the entire mixed area.

[0067] The method for evaluating the standardization of shared bicycles from the perspective of a drone provided by the present invention has the following advantages:

[0068] The present invention relates to the field of smart city management and drone applications, and proposes a method for evaluating the standardization of shared bicycles from the perspective of drones. The method uses drones to inspect shared bicycle parking areas, uses high-definition cameras carried by drones to collect image data in real time, and uses image processing and analysis technology to achieve a comprehensive evaluation of the parking status of shared bicycles. By introducing drone technology, this method realizes automated inspection and evaluation of shared bicycle parking areas, significantly improves evaluation efficiency, provides strong technical support for urban shared bicycle management, helps to improve the city's image, and optimizes citizens' travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0070] Figure 1 It is a schematic diagram of the process of the present invention.

[0071] Figure 2 It is a framework diagram of the neatness evaluation algorithm of the present invention.

[0072] Figure 3 This is a framework diagram of the mixing degree evaluation algorithm of the present invention.

[0073] Figure 4 The confusion matrix diagram provided by the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] The present invention relates to the field of smart city management and drone applications, and proposes a method for evaluating the standardization of shared bicycles from the perspective of drones. The method uses drones to inspect shared bicycle parking areas, uses high-definition cameras carried by drones to collect image data in real time, and uses image processing and analysis technology to achieve a comprehensive evaluation of the parking status of shared bicycles.

[0076] The evaluation process covers three core steps: lodging detection, neatness assessment, and mixed degree assessment. Through advanced image recognition algorithms, the lodging detection module can accurately identify the lodging shared bicycles; the neatness assessment module analyzes the angle information between the arrangement angle of the shared bicycles and the preset parking boundary line based on image processing methods, thereby evaluating the neatness of the bicycle parking; the mixed degree assessment module analyzes the mixed placement of shared bicycles of different colors based on image processing methods, and evaluates the mixed placement of shared bicycles of different brands or types. Finally, the quantitative scores of the lodging degree, neatness, and mixed placement are combined to comprehensively calculate the health assessment results of the shared bicycle parking area.

[0077] By introducing drone technology, this method realizes the automated inspection and assessment of shared bicycle parking areas, significantly improving the assessment efficiency, providing strong technical support for urban shared bicycle management, and helping to enhance the city's image and optimize citizens' travel experience.

[0078] See also Figure 1 The present invention provides a method for evaluating the standardization of shared bicycles from the perspective of a drone, comprising the following steps:

[0079] Step S1: Image acquisition:

[0080] The drone is equipped with a high-definition camera and conducts inspections along the specified route. During the inspection process, it collects images of the shared bicycle parking area in real time.

[0081] This step is as follows: The drone uses a multi-rotor drone with vertical take-off and landing and hovering functions to meet the inspection needs of complex terrain and narrow spaces. The drone is a lightweight multi-rotor drone equipped with a gimbal camera, a high-precision positioning module, an obstacle avoidance sensor, etc. The gimbal camera has wide-angle, zoom and infrared shooting functions, which can realize wide-angle, zoom and infrared shooting functions, and can clearly capture the distribution and status of shared bicycles in the parking area. The cruising flight path of the drone can be flexibly adjusted according to the actual situation of the shared bicycle parking area.

[0082] Step S2: Lodging detection:

[0083] Use image recognition technology to process and analyze the collected images of shared bicycle parking areas to identify fallen shared bicycles;

[0084] This step is specifically as follows: a fallover detection module is used to detect the fallover of shared bicycles. The module uses deep learning technology and can automatically learn and identify shared bicycles and their fallover status by training a large amount of sample data. The algorithm based on deep learning has high accuracy and robustness and can adapt to various complex conditions such as lighting and weather.

[0085] The fall detection module combines GPS information and timestamps to accurately locate the location of the fallen shared bicycle and the time when the fall occurred;

[0086] Specifically, it includes two stages: shared bicycle area interception and lying identification, including the following steps:

[0087] Step S2.1, constructing a training sample set of fallen shared bicycles; the training sample set includes multiple training samples; each of the training samples is a high-quality picture of a fallen shared bicycle; the pictures of fallen shared bicycles include pictures taken from a drone perspective and open source pictures; the pictures of fallen shared bicycles have a minimum circumscribed rectangle of the fallen shared bicycle, forming a labeling box of the fallen shared bicycle;

[0088] Specifically, the images taken by the drone are divided into images of the shared bicycle area, the location of the shared bicycles is determined, and the detection results are passed to the next stage for identification;

[0089] The shared bicycle lying dataset is collected and annotated independently. Low-quality images are eliminated when collecting shared bicycle lying images. The image quality evaluation function is established by combining subjective and objective evaluation:

[0090] Therefore, the fallen shared bicycle image is an image of a fallen shared bicycle whose image quality is higher than a set value after being evaluated by the image quality evaluation function;

[0091] The image quality evaluation function is:

[0092]

[0093] Where: α1, α2, α3 are the proportions of subjective evaluation factors, objective evaluation factors and image peak signal-to-noise ratio, α1<α2<α3, and α1+α2+α3=1; F i is the score given to the image through subjective evaluation for the i-th time, n is the number of subjective evaluations; Z j is the score given to the image through objective evaluation for the jth time, m is the number of objective evaluations; χ represents the peak signal-to-noise ratio of the image;

[0094] Step S2.2, introducing the shared bicycle collapse training sample set into the feature pyramid structure deep learning network of the residual attention mechanism for training, and obtaining a trained collapse detection model;

[0095] Step S2.2.1, pre-processing each shared bicycle lying training sample input into the network and adjusting it to a specified size, for example, 448x448;

[0096] Step S2.2.2, split the preprocessed shared bicycle lying training samples into SxS cells, each cell is used to detect the shared bicycle image whose center point falls on the cell, and each cell obtains the shared bicycle prediction box and its confidence;

[0097] Among them: The confidence of the shared bicycle lying information is calculated using accuracy, precision and recall, such as Figure 4 As shown, all three are calculated through the confusion matrix;

[0098] Step S2.2.3, each cell gives the probability value of its predicted category, denoted as Pr;

[0099] Step S2.2.4, traverse all cells and filter the set of cell center points whose intersection-over-union ratio is greater than the threshold, that is: if the intersection-over-union ratio is greater than the threshold, the cell center point is retained, otherwise, the cell center point is discarded, assuming that a total of w grid center points are obtained;

[0100] In step S2.2.5, the accuracy of the bounding box is expressed by the intersection-over-union ratio between the shared bike prediction box and the actual box, denoted as IOU;

[0101] Step S2.2.6, for the set formed by the w grid center points screened in step S2.2.4, the loss value is calculated using the loss function. If the loss value is less than the set value, the training of the deep learning network is completed, and a trained lodging detection model is obtained, and step S2.3 is executed; otherwise, the parameters of the deep learning network are adjusted according to the loss value to obtain an updated deep learning network. For the updated deep learning network, the remaining shared bicycle lodging training samples are used to continue training; this is continuously iterated until a trained deep learning network is obtained, and step S2.3 is executed;

[0102] Step S2.3, using the trained fall detection model, detect and capture the input shared bicycle parking area image.

[0103] Step S3: Regularity evaluation:

[0104] Based on the detection of falling over, the neatness of the shared bicycle parking area is further evaluated. In the neatness evaluation step, the angle between the line connecting the centroids of each shared bicycle and the reference line can be calculated to further analyze the neatness of shared bicycle parking. This information can provide a reference for optimizing the shared bicycle parking layout.

[0105] Specifically, based on the detection of falling over, the neatness of the shared bicycle parking area is further evaluated: first, the outline of each shared bicycle in the shared bicycle parking area image is extracted through image processing technology, and the center of mass position of each shared bicycle is calculated; then, a point cloud is drawn on the blank image according to the center of mass position of the shared bicycle, and linear fitting is performed, and the neatness is evaluated according to the angle between the parked road edge and the fitted straight line;

[0106] In this step, the neatness evaluation is to calculate the angle between the line connecting the centroids of each shared bicycle and the reference line to analyze the neatness of shared bicycle parking. Specifically, the neatness evaluation module is used to accurately evaluate the neatness of shared bicycle parking. In addition, the neatness evaluation module combines GPS information and timestamps to accurately track the location of each shared bicycle parking area and accurately record the inspection time. Figure 2 As shown, including:

[0107] Step S3.1: Color space conversion and color recognition:

[0108] The shared bicycle parking area image is converted from RGB space to HSV space to better identify and distinguish the colors of shared bicycles from the background.

[0109] In the HSV space, by setting a suitable threshold range, the main color of the shared bicycle is identified and the corresponding color mask image is constructed;

[0110] Step S3.2: Shared bicycle contour extraction and centroid positioning:

[0111] Using image processing technology, the shared bicycle outlines are extracted from the color mask image; for each shared bicycle outline, its centroid position is calculated as the representative point of the shared bicycle in the shared bicycle parking area image;

[0112] Step S3.3: Point cloud generation and linear fitting:

[0113] On the blank image, according to the color and centroid position of the shared bicycle, draw points of the same color at the corresponding centroid position to form a point cloud map;

[0114] According to the distribution of the point cloud in the point cloud map, it is divided into two areas, the upper and lower areas; linear fitting is performed on the point clouds of the overall area and the upper and lower areas respectively, and three fitting lines are obtained, namely: the overall fitting line, the upper half fitting line and the lower half fitting line;

[0115] Step S3.4: Road edge recognition and angle measurement:

[0116] Using image processing technology, the road boundary lines in the shared bicycle parking area image are identified and their directions are determined;

[0117] The angles between the overall fitting line, the upper fitting line, the lower fitting line and the road boundary line are measured respectively; these angles reflect the degree of alignment between the shared bicycle parking and the boundary line.

[0118] Step S3.5: Neatness score calculation:

[0119] According to the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle, the scores of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle are obtained respectively;

[0120] Determine the importance of the overall fitting straight line angle, the upper half fitting straight line angle and the lower half fitting straight line angle in evaluating the neatness of shared bicycles, and assign corresponding weights, which are β1, β2 and β3 respectively. The weights can be adjusted according to actual needs.

[0121] The neatness score N is obtained according to the following formula:

[0122] N=x1×β1+x2×β2+x3×β3

[0123] Where x1, x2, and x3 are the scores of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle, respectively.

[0124] Step S4: Mixing degree evaluation:

[0125] In the mixed degree assessment step, a comprehensive analysis is conducted in combination with the color characteristics of shared bicycles. Clustering algorithms, spatial interpolation and other methods can be used to process and analyze point cloud data to obtain more accurate spatial distribution information of shared bicycles. This comprehensive analysis can improve the accuracy and reliability of mixed degree assessment.

[0126] Evaluate whether there is a mixed placement phenomenon in the shared bicycle parking area image, that is, whether shared bicycles of different types or brands are mixed together: Use color recognition and image segmentation technology to classify shared bicycles in the shared bicycle parking area image by color, and calculate the spatial distribution entropy of each type of shared bicycle in the shared bicycle parking area; Based on the spatial distribution entropy and the area proportion of the mixed placement area, evaluate the mixed placement degree in the shared bicycle parking area;

[0127] In this step, the mixed parking degree assessment module is used to assess whether there are different types or brands of shared bicycles parked mixedly in the shared bicycle parking area image. At the same time, combined with GPS information and timestamps, the location of each shared bicycle parking area is accurately tracked through GPS information, and the exact time of inspection is recorded through timestamps. The specific method is as follows: Figure 3 As shown:

[0128] Step S4.1: Point cloud acquisition and mixed area identification:

[0129] Obtain the point cloud image generated according to the color of the shared bicycles in step S3.3; in the point cloud image, determine whether there is a mixed phenomenon of shared bicycles based on the color information; if so, extract the point cloud data of the mixed area and execute step S4.2;

[0130] Step S4.2: Spatial distribution entropy evaluation:

[0131] The spatial distribution entropy of the point cloud map of the mixed area is evaluated to quantify the degree of mixing. The mixed area is divided into multiple grids, and the spatial distribution entropy of each grid is calculated using the following formula;

[0132]

[0133] Among them, E u is the spatial distribution entropy of the u-th grid, V is the number of colors of shared bicycles, and p uv is the ratio of the number of shared bicycles of the vth color in the uth grid to the total number of shared bicycles in the grid;

[0134] The spatial distribution entropy of each grid in the mixed area is averaged to obtain the average spatial distribution entropy of the whole mixed area E. avg :

[0135] Step S4.3: Quantification and degree assessment of mixed areas:

[0136] Determine the ratio of the mixed area to the total area of ​​shared bicycles, expressed as the mixed area ratio A, and quantify the scope of mixed placement;

[0137] Combined with the area ratio A of the mixed area and the average value E of the spatial distribution entropy of the mixed area as a whole avg , use the following formula to get the mixing degree score M:

[0138] M=ω A ×A+ω E ×E avg

[0139] Where: A is the weight of the area ratio of the mixed area, A is the area ratio of the mixed area; ω E is the weight of the spatial distribution entropy of the entire mixing area, E avg It is the average value of the spatial distribution entropy of the entire mixed area. Weight ω A and ω E It can be adjusted according to actual needs.

[0140] Step S5: Health score:

[0141] The health of the shared bicycle parking area is scored based on the results of lodging detection, neatness assessment, and mixed degree assessment. The method is as follows: the scoring criteria and weights of different indicators are set, and according to the scores of each indicator, the final health score H is obtained using the following formula:

[0142]

[0143] Where: δ1, δ2, δ3 are the proportions of the evaluation factors of falling, neatness and mixed degree, and δ1+δ2+δ3=1; y1 is the number of fallen bicycles, Y is the total number of shared bicycles, N is the neatness score, and M is the mixed degree score.

[0144] Therefore, the calculation of the health assessment results adopts the weighted summation method. First, the corresponding weight values ​​are assigned according to the importance of the degree of lodging, neatness and mixed degree in the assessment. For example, the weight of the degree of lodging is 0.4, the weight of neatness is 0.3, and the weight of mixed degree is 0.3. Then these weight values ​​are multiplied by the corresponding quantitative scores and summed to obtain the final health assessment result. The setting of weight values ​​can be adjusted according to actual needs to meet different assessment needs.

[0145] An example is listed below:

[0146] The drone conducts inspections along the specified route, obtains pictures taken by the drone, and transmits them to the health assessment module;

[0147] The health assessment module includes a lodging detection module, a neatness assessment module and a mixed degree assessment module;

[0148] Lodging detection module:

[0149] The fallen bicycle area image is obtained through the falling detection module to detect whether there are fallen shared bicycles, and the total number of shared bicycles Y and the number of fallen shared bicycles y1 are obtained;

[0150] Falling detection module: Through image recognition technology, it can quickly detect and mark fallen shared bicycles;

[0151] The falling detection module is specially designed to detect whether there is a falling phenomenon in the shared bicycle. It is used to receive the image data transmitted by the drone airport and analyze the image data.

[0152] The algorithm used by the fall detection module is based on image recognition technology to identify whether a shared bicycle has fallen over. In addition, the fall detection module can also combine GPS information and timestamps to accurately locate the location and time of the fallen shared bicycle, providing accurate data support for subsequent maintenance and scheduling.

[0153] Regularity assessment module:

[0154] The neatness assessment module assesses the neatness of the shared bicycle image and obtains the shared bicycle color point cloud image and neatness score N, as follows: Figure 2 As shown;

[0155] The neatness assessment module can receive and process the image data transmitted by the drone airport, so as to accurately assess the neatness of shared bicycle parking. This module uses efficient image processing algorithms to accurately assess the neatness of shared bicycle parking. In addition, the module also combines GPS information and timestamps to ensure accurate tracking of the location of each shared bicycle parking area and accurate recording of inspection time.

[0156] Mixing degree evaluation module:

[0157] The mixed placement evaluation module detects whether there is a mixed placement of different types of shared bicycles based on the point cloud image. If there is a mixed placement, the mixed placement degree is evaluated and a mixed placement score M is obtained; Figure 3 As shown;

[0158] The mixed degree assessment module is designed to assess whether there are mixed parking of different types or brands of shared bicycles in the shared bicycle parking area, so as to improve the orderliness and management efficiency of parking. This module can receive and process the image data transmitted by drones or drone fields, and accurately assess the mixed degree through an efficient algorithm. The algorithm is based on image processing technology and evaluates whether there is mixed parking and its mixed degree based on the previously obtained point cloud map. In order to more accurately assess the mixed degree, the mixed degree assessment module also combines GPS information and timestamps. GPS information ensures accurate tracking of the location of each shared bicycle parking area, while the timestamp records the exact time of the inspection. Provide data support for the management and scheduling of shared bicycles.

[0159] The information obtained from the three modules is comprehensively analyzed to obtain the overall parking health H.

[0160] The present invention provides a method for evaluating the standardization of shared bicycles from the perspective of a drone. By using a drone for regular inspections and a health assessment module to process data, the parking status of shared bicycles can be accurately and efficiently detected, thereby assisting in effectively managing the problem of disorderly parking of shared bicycles.

[0161] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A method for evaluating the standardization of shared bicycles from the perspective of a drone, characterized in that: The following steps are involved: Step S1: Image acquisition: The drone is equipped with a high-definition camera and conducts inspections along the specified route. During the inspection process, it collects images of the shared bicycle parking area in real time. Step S2: Lodging detection: Use image recognition technology to process and analyze the collected images of shared bicycle parking areas to identify fallen shared bicycles; Step S3: Regularity evaluation: Based on the detection of falling over, the neatness of the shared bicycle parking area is further evaluated: first, the outline of each shared bicycle in the shared bicycle parking area image is extracted through image processing technology, and the center of mass position of each shared bicycle is calculated; then, a point cloud is drawn on the blank image according to the center of mass position of the shared bicycle, and a linear fit is performed, and the neatness is evaluated according to the angle between the parking edge and the fitted straight line; Step S4: Mixing degree evaluation: Evaluate whether there is a mixed placement phenomenon in the shared bicycle parking area image, that is, whether shared bicycles of different types or brands are mixed together: Use color recognition and image segmentation technology to classify shared bicycles in the shared bicycle parking area image by color, and calculate the spatial distribution entropy of each type of shared bicycle in the shared bicycle parking area; Based on the spatial distribution entropy and the area proportion of the mixed placement area, evaluate the mixed placement degree in the shared bicycle parking area; Step S5: Health score: The health of the shared bicycle parking area is scored based on the results of lodging detection, neatness assessment, and mixed degree assessment. The method is as follows: the scoring criteria and weights of different indicators are set, and according to the scores of each indicator, the final health score H is obtained using the following formula: Where: δ1, δ2, δ3 are the proportions of the evaluation factors of falling, neatness and mixed degree, and δ1+δ2+δ3=1; y1 is the number of fallen bicycles, Y is the total number of shared bicycles, N is the neatness score, and M is the mixed degree score.

2. According to the method for evaluating the standardization of shared bicycles from the perspective of a drone as described in claim 1, it is characterized in that: Step S1 is specifically as follows: The drone is a multi-rotor drone with vertical take-off and landing and hovering functions. It is equipped with a gimbal camera, a high-precision positioning module, and an obstacle avoidance sensor. The gimbal camera has wide-angle, zoom, and infrared shooting functions.

3. The method for evaluating the standardization of shared bicycles from the perspective of a drone according to claim 1 is characterized in that: In step S2, a fall detection module is used to detect the fall of shared bicycles. The module uses deep learning technology to automatically learn and identify shared bicycles and their fall status by training a large amount of sample data, and accurately locates the position of the fallen shared bicycle and the time when the fall occurred in combination with GPS information and timestamps. Specifically, the module includes two stages: shared bicycle area interception and fall identification, and includes the following steps: Step S2.1, constructing a training sample set of fallen shared bicycles; the training sample set includes multiple training samples; each training sample is a picture of a fallen shared bicycle; the pictures of fallen shared bicycles include pictures taken from a drone perspective and open source pictures; the pictures of fallen shared bicycles have a minimum circumscribed rectangle of the fallen shared bicycle, forming a labeling box of the fallen shared bicycle; The shared bicycle lying down picture is a shared bicycle lying down picture whose image quality is higher than a set value after being evaluated by an image quality evaluation function; The image quality evaluation function is: Where: α1, α2, α3 are the proportions of subjective evaluation factors, objective evaluation factors and image peak signal-to-noise ratio, α1<α2<α3, and α1+α2+α3=1; F i is the score given to the image through subjective evaluation for the i-th time, n is the number of subjective evaluations; Z j is the score given to the image through objective evaluation for the jth time, m is the number of objective evaluations; χ represents the peak signal-to-noise ratio of the image; Step S2.2, introducing the shared bicycle collapse training sample set into the feature pyramid structure deep learning network of the residual attention mechanism for training, and obtaining a trained collapse detection model; Step S2.2.1, pre-processing each shared bicycle lying training sample input into the network and adjusting it to a specified size; Step S2.2.2, split the preprocessed shared bicycle lying training samples into SxS cells, each cell is used to detect the shared bicycle image whose center point falls on the cell, and each cell obtains the shared bicycle prediction box and its confidence; Among them: the confidence of the shared bicycle lying information is calculated by accuracy, precision and recall, all of which are calculated through confusion matrix; Step S2.2.3, each cell gives the probability value of its predicted category, denoted as Pr; Step S2.2.4, traverse all cells and filter the set of cell center points whose intersection-over-union ratio is greater than the threshold, that is: if the intersection-over-union ratio is greater than the threshold, the cell center point is retained, otherwise, the cell center point is discarded, assuming that a total of w grid center points are obtained; In step S2.2.5, the accuracy of the bounding box is expressed by the intersection-over-union ratio between the shared bike prediction box and the actual box, denoted as IOU; Step S2.2.6, for the w grid center points selected in step S2.2.4, use the loss function to calculate the loss value. If the loss value is less than the set value, the training of the deep learning network is completed, and a trained lodging detection model is obtained, and step S2.3 is executed; otherwise, the parameters of the deep learning network are adjusted according to the loss value to obtain an updated deep learning network. For the updated deep learning network, the remaining shared bicycle lodging training samples are used to continue training; this is continuously iterated until a trained deep learning network is obtained, and step S2.3 is executed; Step S2.3, using the trained fall detection model, detect and capture the input shared bicycle parking area image.

4. The method for evaluating the standardization of shared bicycles from the perspective of a drone according to claim 1 is characterized in that: In step S3, the neatness evaluation is as follows: by calculating the angle between the line connecting the centroids of the shared bicycles and the reference line, the neatness of the shared bicycle parking is analyzed.

5. The method for evaluating the standardization of shared bicycles from the perspective of a drone according to claim 4 is characterized in that: Step S3 specifically includes: using the neatness assessment module to accurately assess the neatness of shared bicycle parking, and the neatness assessment module combines GPS information and timestamps to accurately track the location of each shared bicycle parking area and accurately record the inspection time, specifically including: Step S3.1: Color space conversion and color recognition: Convert the shared bicycle parking area image from RGB space to HSV space; in HSV space, identify the color of shared bicycles by setting the threshold range and construct the corresponding color mask image; Step S3.2: Shared bicycle contour extraction and centroid positioning: Using image processing technology, the shared bicycle outlines are extracted from the color mask image; for each shared bicycle outline, its centroid position is calculated as the representative point of the shared bicycle in the shared bicycle parking area image; Step S3.3: Point cloud generation and linear fitting: On the blank image, according to the color and centroid position of the shared bicycle, draw points of the same color at the corresponding centroid position to form a point cloud map; According to the distribution of the point cloud in the point cloud map, it is divided into two areas, the upper and lower areas; linear fitting is performed on the point clouds of the overall area and the upper and lower areas respectively, and three fitting lines are obtained, namely: the overall fitting line, the upper half fitting line and the lower half fitting line; Step S3.4: Road edge recognition and angle measurement: Using image processing technology, the road boundary lines in the shared bicycle parking area image are identified and their directions are determined; The angles between the overall fitting straight line, the upper half fitting straight line, the lower half fitting straight line and the road boundary line are measured respectively; Step S3.5: Neatness score calculation: Determine the importance of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle in evaluating the neatness of shared bicycles, and assign corresponding weights, β1, β2, and β3, respectively: The neatness score N is obtained according to the following formula: N=x1×β1+x2×β2+x3×β3 Where x1, x2, and x3 are the scores of the overall fitting straight line angle, the upper half fitting straight line angle, and the lower half fitting straight line angle, respectively.

6. The method for evaluating the standardization of shared bicycles from the perspective of a drone according to claim 5 is characterized in that: In step S4, a mixed parking degree assessment module is used to assess whether there are different types or brands of shared bicycles parked mixedly in the shared bicycle parking area image. At the same time, the location of each shared bicycle parking area is accurately tracked through GPS information and timestamps, and the exact time of inspection is recorded through timestamps. The specific method is as follows: Step S4.1: Point cloud acquisition and mixed area identification: Obtain the point cloud image generated according to the color of the shared bicycles in step S3.3; in the point cloud image, determine whether there is a mixed phenomenon of shared bicycles based on the color information; if so, extract the point cloud data of the mixed area and execute step S4.2; Step S4.2: Spatial distribution entropy evaluation: The mixed area is divided into multiple grids, and the spatial distribution entropy of each grid is calculated using the following formula; Among them, E u is the spatial distribution entropy of the u-th grid, V is the number of colors of shared bicycles, and p uv is the ratio of the number of shared bicycles of the vth color in the uth grid to the total number of shared bicycles in the grid; The spatial distribution entropy of each grid in the mixed area is averaged to obtain the average spatial distribution entropy of the whole mixed area E. avg : Step S4.3: Quantification and degree assessment of mixed areas: Determine the ratio of the mixed area to the total area of ​​shared bicycles, expressed as the mixed area area ratio A: Combined with the area ratio A of the mixed area and the average value E of the spatial distribution entropy of the mixed area as a whole avg , use the following formula to get the mixing degree score M: M=ω A ×A+ω E ×E avg Where: A is the weight of the area ratio of the mixed area, A is the area ratio of the mixed area; ω E is the weight of the spatial distribution entropy of the entire mixing area, E avg It is the average value of the spatial distribution entropy of the entire mixed area.

Images