A street lamp damage detection method, device and related components

Abstract

The application discloses a street lamp damage detection method and device and related components, the method comprising: acquiring picture data; inputting the picture data into a target detection model to obtain a lamp pole detection frame; inputting the coordinates of the center point of the lamp pole detection frame into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection frame and a light source; obtaining the search range of the light source according to the center point of the lamp pole detection frame and the distance; performing gray processing and light source detection on the picture data in the search range to obtain coordinate information; determining whether the displacement of the light source is greater than a displacement threshold according to the coordinate information in adjacent two frames of picture data; if yes, the light source is another light source; if no, determining whether the gray value of the coordinate information in continuous multiple frames of picture data is greater than or equal to a light source threshold; if yes, the street lamp is in a normal state; if no, the street lamp is in a damaged state. The application realizes the detection of damaged street lamps by the method, without the need for manual work and additional sensor equipment, and the cost is low.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of lighting technology, in particular to a street lamp damage detection method and device and related components. BACKGROUND

[0002] As an important facility for urban road lighting, street lamps play a crucial role in ensuring the safety of pedestrians and vehicles. However, due to long-term use, adverse weather, traffic accidents, and other reasons, street lamps are prone to damage or failure.

[0003] Currently, most cities usually use the following two methods to detect whether the street lamp is damaged:

[0004] The first method is to use manual patrol to detect the damage of street lamps. This method has the problems of low efficiency, high cost, and missed detection.

[0005] The second method is to use Internet of Things technology to detect street lamp damage information using sensors and other multi-source devices, and transmit the information to the cloud platform to realize monitoring and maintenance. This method has the problems of high cost and maintenance trouble. SUMMARY

[0006] The purpose of the present application is to provide a street lamp damage detection method, device and related components, which aims to solve the problem of high cost of existing street lamp detection methods.

[0007] In a first aspect, the present application provides a street lamp damage detection method, comprising:

[0008] Obtaining video stream data and performing frame extraction processing on the video stream data to obtain picture data;

[0009] Inputting the picture data into a target detection model to obtain a lamp pole detection box;

[0010] Inputting the coordinates of the center point of the lamp pole detection box into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection box and the light source;

[0011] Taking the center point of the lamp pole detection box as the center and the distance as the radius to obtain the search range of the light source;

[0012] Performing grayscale processing and light source detection on the picture data in the search range to obtain the coordinate information of the light source;

[0013] According to the coordinate information of the light source in the adjacent two frames of picture data, it is judged whether the displacement of the light source is greater than a displacement threshold;

[0014] If yes, the light source is determined as other light source; if no, the light source is determined as street lamp and whether the gray value located at the coordinate information in picture data of continuous multiple frames is greater than or equal to a light source threshold value is counted;

[0015] If yes, it is determined that the street lamp is in normal state; if no, it is determined that the street lamp is in damaged state.

[0016] In a second aspect, an embodiment of the present application provides a street lamp damage detection device, comprising:

[0017] A frame extraction unit is configured to acquire video stream data and perform frame extraction processing on the video stream data to obtain picture data.

[0018] A lamp pole detection frame acquisition unit is configured to input the picture data into a target detection model to obtain a lamp pole detection frame.

[0019] A distance acquisition unit is configured to input coordinates of a center point of the lamp pole detection frame into an XGBoost algorithm prediction model to obtain a distance between the center point of the lamp pole detection frame and a light source.

[0020] A search range acquisition unit is configured to take the center point of the lamp pole detection frame as a center and the distance as a radius to obtain a search range of the light source.

[0021] A coordinate information acquisition unit is configured to perform gray processing and light source detection on the picture data in the search range to obtain coordinate information of the light source.

[0022] A displacement judgment unit is configured to judge whether a displacement of the light source is greater than a displacement threshold value according to coordinate information of the light source in adjacent two frames of picture data.

[0023] A light source determination unit is configured to, if yes, determine the light source as other light source; if no, determine the light source as street lamp and count whether a gray value located at the coordinate information in picture data of continuous multiple frames is greater than or equal to a light source threshold value.

[0024] A street lamp damage determination unit is configured to, if yes, determine that the street lamp is in normal state; if no, determine that the street lamp is in damaged state.

[0025] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the street lamp damage detection method of the first aspect when executing the computer program.

[0026] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the street lamp damage detection method in the first aspect.

[0027] The present application discloses a street lamp damage detection method, device and related components, the method comprising: obtaining video stream data and performing frame extraction processing on the video stream data to obtain picture data; inputting the picture data into a target detection model to obtain a lamp pole detection box; inputting the coordinates of the center point of the lamp pole detection box into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection box and the light source; taking the center point of the lamp pole detection box as the center and the distance as the radius to obtain the search range of the light source; performing grayscale processing and light source detection on the picture data in the search range to obtain the coordinate information of the light source; determining whether the displacement of the light source in the coordinate information of the light source in the adjacent two frames of picture data is greater than a displacement threshold; if yes, determining the light source as another light source; if no, determining the light source as a street lamp and counting whether the grayscale value in the coordinate information in the continuous multiple frames of picture data is greater than or equal to a light source threshold; if yes, determining that the street lamp is in a normal state; if no, determining that the street lamp is in a damaged state. The present application realizes automatic detection of damaged street lamps through the above method, has high detection efficiency and accuracy, reduces labor costs, does not need to add new sensor equipment, saves additional costs, can detect street lamps in a large range, and improves the work efficiency of background staff. The embodiments of the present application also provide a street lamp damage detection device, a computer readable storage medium and a computer device, which have the above beneficial effects, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0029] Figure 1 The flowchart of the street lamp damage detection method of the present embodiment;

[0030] Figure 2 The flowchart of the street lamp damage detection method of a specific application scenario;

[0031] Figure 3 The schematic block diagram of the street lamp damage detection device of the present embodiment. DETAILED DESCRIPTION

[0032] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the scope of the present application.

[0033] It should be understood that the terms "comprise" and "include" as used in the specification and the appended claims indicate the presence of the described features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0034] It should also be understood that the terms used in the present application specification are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0035] It should be further understood that the term "and / or" as used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0036] Please refer to Figure 1 and Figure 2 , the present application provides a street lamp damage detection method, comprising:

[0037] S101: acquiring video stream data and performing frame extraction processing on the video stream data to obtain picture data;

[0038] The video stream data in the present embodiment is mainly acquired by the camera arranged on the road, and a single camera can acquire the condition information of multiple street lamps, so that only part of the video stream data of the camera needs to be called, which greatly reduces the complexity and cost of data transmission, and also improves the efficiency of data processing. Therefore, the condition information of multiple street lamps acquired by a single camera makes the system more flexible and efficient in processing a large amount of video stream data, thereby improving the performance and reliability of the entire system.

[0039] S102: inputting the picture data into a target detection model to obtain a lamp pole detection frame;

[0040] Specifically, the target detection model is trained, and the loss value and weight of the model are recorded after each training is completed to obtain a training weight file; then the training weight file is loaded to the target detection model, and the picture data is input to the loaded target detection model to obtain a temporary lamp pole detection frame; then the temporary lamp pole detection frame is subjected to non-maximum suppression filtering processing, and a detection frame with a confidence greater than a set threshold is output to obtain a final lamp pole detection frame, wherein the training weight file is the best training weight file selected; and through the non-maximum suppression processing and the set threshold, the detection frame with low confidence and possible false detection can be effectively filtered out, and the accuracy of the detection result is improved.

[0041] In this embodiment, the target detection model needs to create a data set during training. Specifically, according to the shape characteristics of the lamp pole, the lamp pole on the road in the picture data is image-labeled by using an image labeling software to obtain labeling information and save the labeling information into a file, and then the picture data and the corresponding labeling information are divided into a training set, a validation set and a test set according to a predetermined ratio. Labeling the picture data can help the model better understand the lamp pole in the picture and improve the recognition accuracy, and can ensure the integrity and traceability of the labeling information, facilitating subsequent processing and analysis. The labeling information mainly includes position information and category information of the lamp pole.

[0042] In a specific application scenario, according to the shape characteristics of the lamp pole, the lamp pole on the road in the picture data is image-labeled by using a labelme image labeling software to obtain labeling information and save the labeling information into a txt file, and then the picture data and the corresponding labeling information are divided into a training set, a validation set and a test set according to a ratio of 70%, 20% and 10%.

[0043] In this embodiment, the target detection model adopts a Yolov5 network model, which mainly consists of a backbone network, a Neck network and a prediction layer. The backbone network consists of a Focus module, a CBL module, a CSP module and an SPP module. The Focus module is to sample the input image at equal distances, and then perform channel splicing. The CBL module refers to a packaged convolution module, which consists of a convolution (Conv) layer, a normalization (BN) layer and an activation function (Leaky ReLU). The CSP module consists of a CBL module, a residual module and a convolution layer. The SPP module uses 1x1, 5x5, 9x9 and 13x13 maximum pooling respectively. The Neck network is a pyramid structure based on the FPN framework. The prediction layer outputs the category information, coordinate information and confidence of the target object.

[0044] Further, the following loss function is selected in this embodiment to calculate the loss value:

[0045]

[0046] wherein, L represents a loss value; L conf represents a target detection category loss value; L loc represents a detection frame offset regression loss value; N represents the number of all detection frames output by the network; and a represents a constant.

[0047] L loc The calculation formula is:

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] wherein, pos represents a true sample; m is a general term, which respectively represents cx, cy, w, and h in the formula; for example, when calculating the horizontal coordinate cx of the center point of the detection frame, m=cx, at this time, the formula L Loc will be represented as cx represents the horizontal coordinate of the center point of the detection frame, cy represents the vertical coordinate of the center point of the detection frame, w represents the width of the detection frame, and h represents the height of the detection frame. represents whether the i-th predicted frame and the j-th true frame match successfully with respect to category k, when is 1, indicating that the match with the true frame is successful, when is 0, indicating that the match with the true frame is unsuccessful; smooth L1 represents a smooth L1 loss function; represents predicted frame information; represents true frame information; represents the center point horizontal coordinate information of the j-th true frame; represents the center point horizontal coordinate value of the j-th true frame; represents the i-th detection frame

[0055] corresponding center point horizontal coordinate; represents the i-th detection frame corresponding width parameter value; center point vertical coordinate information of the j-th true frame; represents the center point vertical coordinate value of the j-th true frame; represents the i-th detection frame corresponding center point vertical coordinate; a height parameter value corresponding to the i-th detection frame; center point width information of the j-th real frame; a center point width value of the j-th real frame; a width parameter value corresponding to the j-th real frame; center point height information of the j-th real frame; a center point height value of the j-th real frame; a height parameter value corresponding to the j-th real frame; x represents the difference between the prediction frame and the groud truth.

[0056] L conf The calculation formula is:

[0057]

[0058]

[0059] wherein, indicates whether the i-th prediction frame and the j-th real frame match successfully with respect to the class p, when is 1, indicating that the real frame is matched successfully, when is 0, indicating that the real frame is not matched successfully; Softmax value of the i-th candidate frame belonging to the target probability; Neg represents a negative sample; Softmax value of the i-th candidate frame belonging to the background probability; indicates the probability value of the i-th candidate frame in the picture belonging to the target.

[0060] In the embodiment, the non-maximum suppression filtering processing of the temporary lamp pole detection frame includes: for each temporary lamp pole detection frame, calculating the intersection over union (IoU) with all other temporary lamp pole detection frames; if the intersection over union of a certain temporary lamp pole detection frame and another temporary lamp pole detection frame is greater than a set threshold, it is considered that the two temporary lamp pole detection frames repeatedly identify an object, and the score of the temporary lamp pole detection frame with lower probability (i.e. the temporary lamp pole detection frame with smaller intersection over union) is suppressed; the above process is repeated until no temporary lamp pole detection frame is left; the final lamp pole detection frame is the temporary lamp pole detection frame after the non-maximum suppression processing; wherein, there are two ways of suppression, one is “soft non-maximum suppression”, which retains the temporary lamp pole detection frame but reduces its score, which can retain more recall rate, but may sacrifice certain accuracy; the other is “hard non-maximum suppression”, which directly filters out the temporary lamp pole detection frame, which retains relatively less temporary lamp pole detection frame, but usually has higher accuracy.

[0061] When the lamp pole detection frame is obtained through step S102, step S103 is performed.

[0062] S103: input the coordinates of the center point of the lamp pole detection frame into the XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection frame and the light source;

[0063] Before using the XGBoost algorithm prediction model for prediction, the XGBoost algorithm prediction model needs to be trained. The training process of the XGBoost algorithm prediction model includes: inputting sample data into the target detection model to obtain lamp pole sample detection frames; then calculating the distance between the center point of the lamp pole sample detection frame and the light source using the Euclidean distance formula; then taking the coordinates of the center point of the lamp pole sample detection frame as the features of the data set and taking the distance between the center point of the lamp pole sample detection frame and the light source as the label to obtain the data set; then inputting the data set into the XGBoost algorithm prediction model for training and calculating the corresponding objective function; when the training times of the XGBoost algorithm prediction model reach the predetermined training times, the XGBoost algorithm prediction model corresponding to the minimum value of the objective function is obtained.

[0064] Specifically, sample data is input into the target detection model to obtain lamp pole sample detection frames; the distance between the center point of the lamp pole sample detection frame and the light source is calculated using the Euclidean distance formula, which is as follows:

[0065]

[0066] Wherein, L represents the distance between the light and the street lamp, x1 and y1 represent the horizontal and vertical coordinates of the center point of the lamp pole sample detection frame respectively; x2 and y2 represent the horizontal and vertical coordinates of the street lamp respectively.

[0067] Then, the coordinates of the center point of the lamp pole sample detection frame are taken as the features of the data set and the distance between the center point of the lamp pole sample detection frame and the light source is taken as the label to construct the data set of the XGBoost algorithm prediction model. This data set is a supervised data set; then the data set is divided into a training set, a validation set and a test set according to the proportions of 70%, 20% and 10%; then the training set is input into the XGBoost algorithm prediction model for training and the corresponding objective function is calculated.

[0068] Wherein, the XGBoost algorithm prediction model of the present embodiment is a prediction model, and the target is to obtain an optimal objective function F, so that the input horizontal and vertical coordinates (X) of the center point of the lamp pole sample detection frame and the output distance (Y) between the street lamp and the lamp pole achieve the best regression effect; the objective function includes a training error and a regularization term, and the formula is as follows:

[0069] F(θ)=R(θ)+ψ(θ)

[0070]

[0071]

[0072] where F(θ) represents the objective function; R(θ) represents the training error function; ψ(θ) represents the regularization term; n represents the number of samples; y i represents the i-th real value; represents the i-th predicted value; K represents the K-th base evaluator; f k (x i ) represents the prediction function of the K-th base evaluator; x i represents the i-th input value x (i.e., the horizontal and vertical coordinates of the center point of the input light pole sample bounding box); F represents the feature space, f k represents the feature of the K-th base evaluator;

[0073] Assuming that at the t-th step, the predicted value is calculated according to the formula:

[0074]

[0075]

[0076]

[0077]

[0078] where this process is the training process of the explanation model, represents that the initial prediction result is equal to 0; represents the predicted value of the first stage; f1(x i ) represents the prediction function of the first base evaluator; represents the predicted value of the second stage; f2(x i ) represents the prediction function of the second base evaluator; represents the predicted value of the t-th stage; represents the predicted value of the t-1-th stage; f t (x i ) represents the prediction function of the t-th base evaluator.

[0079] The calculation formula of the predicted value is substituted into R(θ), and Taylor expansion is performed, and the objective function can be written as:

[0080]

[0081] where F (t) represents the objective function of the t-th time; y i represents the i-th real value; represents the prediction value of the i-th at the t-1 time; x i represents the input value of the i-th; f t (x i ) represents the prediction function of the t-th base evaluator; n represents the number of samples; ψ(f t ) represents the regularization term.

[0082] When the number of training times of the XGBoost algorithm prediction model reaches the predetermined number of training times, the XGBoost algorithm prediction model corresponding to the minimum value of the objective function is obtained, wherein the XGBoost algorithm prediction model corresponding to the minimum value of the objective function is the best trained model.

[0083] When the best XGBoost algorithm prediction model is obtained, only the coordinates of the center point of the lamp pole detection frame need to be input into the XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection frame and the light source.

[0084] S104: Taking the center point of the lamp pole detection frame as the center and the distance as the radius, the search range of the light source is obtained;

[0085] The purpose of setting the search range is to reduce the influence of other light sources on the image data processing process; Specifically, setting the search range as the area between the street lamp and the lamp pole can effectively filter out the interference that may come from other light sources; These interferences come from the light sources in the surrounding environment, such as buildings, vehicles or other artificial light sources, etc.; In this way, it can be ensured that the image data is only affected by the street lamp light source, so that the street lamp light source detection is more accurate. In addition, using this specific search range for processing can not only improve the accuracy of street lamp light source detection, but also improve the efficiency of the entire calculation process; Because the interference of irrelevant light sources is reduced, the data processing process will be faster and more effective; In this way, more accurate results can be obtained in a shorter time, thereby improving work efficiency.

[0086] S105: Perform grayscale processing and light source detection on the picture data in the search range to obtain the coordinate information of the light source;

[0087] Specifically, the pixel values of the images within the search range are retained, and the pixel values of the images outside the search range are set to 0; Then the picture data is subjected to grayscale processing to obtain grayscale picture data; Then the grayscale values in the grayscale picture data are counted, and the light source threshold is calculated using the Otsu threshold method; Then the grayscale picture data is subjected to binaryzation processing and image morphological operation using the light source threshold to obtain a plurality of light source points; Then the coordinate information of the center point in the plurality of light source points is taken as the coordinate information of the light source.

[0088] In this embodiment, the picture data is mainly processed according to the following formula:

[0089]

[0090] Wherein, Q represents the gray value, R, G, B represent the three primary colors of the image, i.e. the values of red, green and blue.

[0091] In this embodiment, the Otsu threshold value is based on a given threshold T, and the image is divided into background and foreground. The proportion of the background points in the image is p0, and the average gray value is m0. The proportion of the target points in the image is p1, and the average gray value is m1.

[0092] The gray average value m of the whole image is calculated using the following formula:

[0093] m = p0m0 + p1m1

[0094] The variance σ of the gray value 2 is calculated using the following formula:

[0095] σ 2 = p0(m0-m) 2 + p1(m1-m) 2

[0096] The formula of the gray average value is brought into the formula of the variance to obtain the following formula:

[0097] σ 2 = p0p1(m0-m1) 2

[0098] By traversing all the gray values, when the variance value is the maximum value, the corresponding gray value is taken as the light source threshold value.

[0099] In this embodiment, the light source threshold value is used to perform binaryzation processing and image morphological operation on the gray picture data to obtain a plurality of light source points, including:

[0100] The pixel value of the pixel point in the gray picture data whose gray value is greater than the light source threshold value is set to 255, and the pixel value of the pixel point in the gray picture data whose gray value is less than the light source threshold value is set to 0, to obtain binaryzation picture data;

[0101] A cross-shaped structural element is defined, and the center of the structural element is compared with the pixel at the corresponding position in the binaryzation picture data;

[0102] If all the pixels in the structural element match the pixel at the corresponding position in the binaryzation picture data, the corresponding position in the binaryzation picture data is set as the foreground;

[0103] If any pixel in the structure element does not match the pixel at the corresponding position in the binary image data, the corresponding position in the binary image data is set as background.

[0104] The pixels with a value of 255 in the adjacent pixels in the foreground and the background are obtained and taken as light source points.

[0105] In this embodiment, the grayscale image data is binarized by using the light source threshold value, different regions in the image can be distinguished, the contrast of the image is improved, and the light source is more prominent; the grayscale image data after the binarization is subjected to the image morphological operation to divide the foreground and the background, which can effectively remove the noise and burrs in the image after the binarization, and can also smooth the image, further enhancing the quality and the definition of the image.

[0106] S106: determining whether the displacement of the light source is greater than a displacement threshold value according to the coordinate information of the light source in the adjacent two frames of image data;

[0107] Specifically, the coordinate information of the light source in the search range is statistically marked, and then the displacement of the light source is calculated by using a displacement calculation formula, which is as follows:

[0108]

[0109] wherein H represents the distance between the light sources in the adjacent two frames of image data, a1 and b1 represent the horizontal coordinate and the vertical coordinate of the light source of the previous frame of street lamp respectively, and a2 and b2 represent the horizontal coordinate and the vertical coordinate of the light source of the next frame of street lamp respectively.

[0110] After obtaining the displacement of the light source in the adjacent two frames of image data according to the above formula, it is determined whether the displacement of the light source is greater than a displacement threshold value, if yes, step S107 is executed, if not, step S108 is executed, and then step S109 is executed; wherein the displacement threshold value can be set according to the actual application scenario.

[0111] Determining whether the displacement of the light source is greater than a displacement threshold value according to the coordinate information of the light source in the adjacent two frames of image data is mainly to determine whether the light source is a moving light source, if the displacement of the light source is greater than the displacement threshold value, it indicates that the light source is a moving light source, and therefore it can be determined as other light source; if the displacement of the light source is less than or equal to the displacement threshold value, it indicates that the light source is a static light source, and therefore it can be determined as street lamp.

[0112] S107: determining the light source as other light source;

[0113] S108: determining the light source as street lamp;

[0114] S109: whether the gray value of the coordinate information in the picture data of the continuous multiple frames is greater than or equal to the light source threshold value is counted;

[0115] Specifically, if the gray value of the coordinate information is greater than or equal to the light source threshold value, step S110 is performed; if the gray value of the coordinate information is less than the light source threshold value, it indicates that the light source disappears, step S111 is performed, and corresponding alarm information is generated.

[0116] S110: determining that the street lamp is in a normal state;

[0117] S111: determining that the street lamp is in a damaged state.

[0118] The embodiment can construct an automatic street lamp detection method by combining a target detection model, an XGBoost algorithm prediction model, and an image preprocessing method, can monitor the damage of the street lamp in real time, does not need to add a sensor device, saves the cost, can be widely covered, and improves the work efficiency of the background staff.

[0119] Please refer to Figure 3 The embodiment provides a street lamp damage detection device 300, which comprises:

[0120] The frame extraction unit 301 is configured to acquire video stream data and perform frame extraction processing on the video stream data to obtain picture data.

[0121] The lamp pole detection frame acquisition unit 302 is configured to input the picture data into a target detection model to obtain a lamp pole detection frame.

[0122] The distance acquisition unit 303 is configured to input the coordinates of the center point of the lamp pole detection frame into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection frame and a light source.

[0123] The search range acquisition unit 304 is configured to take the center point of the lamp pole detection frame as the center and the distance as the radius to obtain the search range of the light source.

[0124] The coordinate information acquisition unit 305 is configured to perform gray processing and light source detection on the picture data in the search range to obtain the coordinate information of the light source.

[0125] The displacement judgment unit 306 is configured to determine whether the displacement of the light source in the coordinate information in the picture data of adjacent two frames is greater than a displacement threshold value.

[0126] The light source determination unit 307 is configured to determine the light source as another light source if yes; and determine the light source as a street lamp and count whether the gray value of the coordinate information in the picture data of continuous multiple frames is greater than or equal to the light source threshold value if no.

[0127] The street lamp damage determination unit 308 is configured to determine that the street lamp is in a normal state if yes, and determine that the street lamp is in a damaged state if no.

[0128] Further, the lamp pole detection frame acquisition unit 302 comprises:

[0129] The training weight file acquisition subunit is configured to train the target detection model and record the loss value and the weight of the model after each training is completed, to obtain a training weight file.

[0130] The loading subunit is configured to load the training weight file to the target detection model and input the picture data to the loaded target detection model, to obtain a temporary lamp pole detection frame.

[0131] The filtering processing subunit is configured to perform non-maximum suppression filtering processing on the temporary lamp pole detection frame, output a detection frame with a confidence value greater than a set threshold, and obtain a final lamp pole detection frame.

[0132] The calculation formula of the loss value is as follows:

[0133]

[0134] L represents the loss value; L conf represents a target detection category loss value; L loc represents a detection frame offset regression loss value; N represents the number of all detection frames output by the network; and α represents a constant.

[0135] Further, the distance acquisition unit 303 comprises:

[0136] The input subunit is configured to input sample data to the target detection model, to obtain a lamp pole sample detection frame.

[0137] The distance calculation subunit is configured to calculate the distance between the center point of the lamp pole sample detection frame and the light source by using a Euclidean distance formula.

[0138] The data set acquisition subunit is configured to take the coordinates of the center point of the lamp pole sample detection frame as the features of a data set and take the distance between the center point of the lamp pole sample detection frame and the light source as a label, to obtain the data set.

[0139] The target function calculation subunit is configured to input the data set to the XGBoost algorithm prediction model for training and calculate a corresponding target function.

[0140] The model obtaining subunit is configured to obtain the XGBoost algorithm prediction model corresponding to the minimum target function when the number of training times of the XGBoost algorithm prediction model reaches a predetermined number of training times.

[0141] The target function is:

[0142]

[0143] F (t) represents the target function of the tth time; y i represents the true value of the ith time; represents the prediction value of the (t-1)th time of the ith time; x i represents the input value of the ith time; f t (x i ) represents the prediction function of the tth base evaluator; n represents the number of samples; and ψ(f t ) represents a regularization term.

[0144] Further, the coordinate information obtaining unit 305 comprises:

[0145] The pixel value setting subunit is configured to retain the pixel value of the image within the search range and set the pixel value of the image outside the search range to 0.

[0146] The grayscale processing subunit is configured to perform grayscale processing on the picture data to obtain grayscale picture data.

[0147] The statistical subunit is configured to count the grayscale values in the grayscale picture data and calculate a light source threshold value by using the Otsu threshold method.

[0148] The light source point obtaining subunit is configured to perform binaryzation processing and image morphological operation on the grayscale picture data by using the light source threshold value to obtain a plurality of light source points.

[0149] The light source coordinate information obtaining subunit is configured to take the coordinate information of the center point in the plurality of light source points as the coordinate information of the light source.

[0150] Further, the light source point obtaining subunit is further configured to:

[0151] set the pixel value of the pixel point with a grayscale value greater than the light source threshold value in the grayscale picture data to 255 and set the pixel value of the pixel point with a grayscale value less than the light source threshold value in the grayscale picture data to 0 to obtain binaryzation picture data.

[0152] define a cross-shaped structural element and compare the center of the structural element with the pixel at the corresponding position in the binaryzation picture data.

[0153] if all pixels in the structure element match the pixels at the corresponding positions in the binary image data, then the corresponding positions in the binary image data are set to foreground;

[0154] if any pixel in the structure element does not match the pixel at the corresponding position in the binary image data, then the corresponding position in the binary image data is set to background;

[0155] obtaining the pixel points with a pixel value of 255 in the foreground and the background, and taking the pixel points with a pixel value of 255 in the foreground and the background as light source points.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0157] The application further provides a computer readable storage medium, which has a computer program stored thereon, and the computer program can implement the method provided in the foregoing embodiments when executed. The storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0158] The application further provides a computer device, which can include a memory and a processor, the memory has a computer program stored therein, and the processor can implement the method provided in the foregoing embodiments when calling the computer program in the memory. Of course, the computer device can further include various network interfaces, power supplies and other components.

[0159] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part. It should be pointed out that, for those skilled in the art, without departing from the principle of the application, some improvements and modifications can be made to the application, and these improvements and modifications also fall within the protection scope of the claims of the application.

[0160] It should also be noted that, in the specification, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0161] Without further limitation, an element defined by an "includes a... " does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A street light damage detection method characterized by, The method comprises the following steps: acquiring video stream data and performing frame extraction on the video stream data to obtain picture data; inputting the picture data into a target detection model to obtain a lamp pole bounding box; inputting the coordinates of the center point of the lamp pole bounding box into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole bounding box and a light source; taking the center point of the lamp pole bounding box as the center and the distance as the radius to obtain a search range of the light source; performing grayscale processing and light source detection on the picture data in the search range to obtain coordinate information of the light source; judging whether the displacement of the light source in adjacent two frames of picture data is greater than a displacement threshold value according to the coordinate information of the light source; if yes, determining the light source as another light source; if no, determining the light source as a street lamp and judging whether the grayscale value of the light source in continuous multiple frames of picture data is greater than or equal to a light source threshold value; if yes, determining that the street lamp is in a normal state; if no, determining that the street lamp is in a damaged state.

2. The street light damage detection method according to claim 1, characterized by, The step of inputting the picture data into the target detection model to obtain a lamp pole bounding box comprises the following steps: training the target detection model and recording the loss value and weight of the model after each training to obtain a training weight file; loading the training weight file into the target detection model and inputting the picture data into the loaded target detection model to obtain a temporary lamp pole bounding box; performing non-maximum suppression filtering processing on the temporary lamp pole bounding box to output a bounding box with a confidence value greater than a set threshold value to obtain a final lamp pole bounding box.

3. The street light damage detection method according to claim 2, characterized by, The calculation formula of the loss value is: wherein L represents a loss value; L conf represents a target detection class loss value; L loc represents a detection frame offset regression loss value; N represents a number of all detection frames output by the network; and a represents a constant.

4. The street light damage detection method according to claim 1, characterized by, The training process of the XGBoost algorithm prediction model comprises the following steps: inputting sample data into a target detection model to obtain a lamp pole sample bounding box; calculating the distance between the center point of the lamp pole sample bounding box and a light source by using a Euclidean distance formula; taking the coordinates of the center point of the lamp pole sample bounding box as features of a data set and taking the distance between the center point of the lamp pole sample bounding box and the light source as a label to obtain the data set; inputting the data set into the XGBoost algorithm prediction model for training and calculating a corresponding objective function; when the training number of the XGBoost algorithm prediction model reaches a predetermined training number, obtaining the XGBoost algorithm prediction model corresponding to the minimum value of the objective function.

5. The street light damage detection method according to claim 4, characterized by, The objective function is: where F (t) represents the objective function at the tth time; y i represents the true value at the ith time; represents the predicted value at the (t-1)th time at the ith time; x i represents the input value at the ith time; f t (x i ) represents the prediction function of the tth base evaluator; n represents the number of samples; ψ(f t ) represents the regularization term.

6. The street light damage detection method according to claim 1, characterized by, The step of performing grayscale processing and light source detection on the picture data in the search range to obtain coordinate information of the light source comprises the following steps: retaining the pixel values of images in the search range and setting the pixel values of images outside the search range to 0; performing grayscale processing on the picture data to obtain grayscale picture data; statistically analyzing the grayscale values in the grayscale picture data and calculating a light source threshold value by using the Otsu threshold method; performing binaryzation processing and image morphological operation on the grayscale picture data by using the light source threshold value to obtain a plurality of light source points; taking the coordinate information of the center point of the plurality of light source points as the coordinate information of the light source.

7. The street light damage detection method according to claim 6, characterized by, The binarization processing and image morphological operation of the gray picture data by using the light source threshold value include: Setting the pixel value of the pixel point with the gray value greater than the light source threshold value in the gray picture data to 255, and setting the pixel value of the pixel point with the gray value less than the light source threshold value in the gray picture data to 0, to obtain binary picture data; Defining a cross-shaped structure element, and comparing the center of the structure element with the pixel at the corresponding position in the binary picture data; If all the pixels in the structure element match the pixel at the corresponding position in the binary picture data, the corresponding position in the binary picture data is set as foreground; If any pixel in the structure element does not match the pixel at the corresponding position in the binary picture data, the corresponding position in the binary picture data is set as background; Obtaining the pixel point with the adjacent pixel value of 255 in the foreground and the background, and taking the pixel point with the adjacent pixel value of 255 in the foreground and the background as a light source point.

8. A street light damage detection apparatus characterized by comprising: It includes: An extracting unit configured to obtain video stream data and perform frame extraction on the video stream data to obtain picture data; A lamp pole detection box obtaining unit configured to input the picture data into a target detection model to obtain a lamp pole detection box; A distance obtaining unit configured to input the coordinates of the center point of the lamp pole detection box into an XGBoost algorithm prediction model to obtain the distance between the center point of the lamp pole detection box and a light source; A search range obtaining unit configured to take the center point of the lamp pole detection box as the center and the distance as the radius to obtain the search range of the light source; A coordinate information obtaining unit configured to perform gray processing and light source detection on the picture data in the search range to obtain the coordinate information of the light source; A displacement judging unit configured to judge whether the displacement of the light source is greater than a displacement threshold value according to the coordinate information of the light source in adjacent two frames of picture data; A light source judging unit configured to, if yes, judge the light source as another light source; If no, judge the light source as a street lamp and count whether the gray value at the coordinate information in the picture data of continuous multiple frames is greater than or equal to a light source threshold value; A street lamp damage judging unit configured to, if yes, judge that the street lamp is in a normal state; If no, judge that the street lamp is in a damaged state.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the street lamp damage detection method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program causes the processor to execute the street lamp damage detection method in any one of claims 1 to 7 when the processor executes the computer program.

Images