Overhead line spark detection method and device, computer equipment, readable storage medium and program product

Abstract

The application relates to an overhead line spark detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring to-be-processed multimedia data, the to-be-processed multimedia data comprising at least one to-be-processed multimedia data frame; each to-be-processed multimedia data frame comprising a structure reference point; identifying each to-be-processed multimedia data frame to obtain a to-be-confirmed spark, acquiring the position information of the to-be-confirmed spark in each to-be-processed multimedia data frame, and generating trajectory data according to the position information; when the trajectory data is in a confirmation state, acquiring a first structure near neighbor score corresponding to the trajectory data; acquiring an interference counter-evidence score corresponding to the to-be-confirmed spark, and when it is determined that the to-be-confirmed spark is a target spark according to the first structure near neighbor score and the interference counter-evidence score, generating an alarm prompt information. The method can improve the spark detection rate.

Description

Technical Field

[0001] This application relates to the field of smart grid technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting sparks on overhead lines. Background Technology

[0002] During the inspection of overhead transmission lines, distribution lines, and external equipment of substations, abnormal phenomena such as sparks, discharges, and arcs are usually characterized by short durations, rapid changes in brightness, and localized locations.

[0003] In related technologies, target detection models are generally used to process patrol videos frame by frame, and the presence of sparks is determined based on the single-frame detection confidence threshold. However, the single-frame detection method suffers from jitter in the detection results, and the spark detection results are not accurate enough.

[0004] In related technologies, there are also methods that add simple cross-frame association mechanisms on the basis of single-frame detection results. Although this type of method can reduce the problem of single-frame detection jitter to a certain extent, it still mainly relies on the continuity of candidate boxes in the pixel coordinate system and the detection confidence. It lacks a comprehensive judgment on the structure of overhead line circuit equipment, camera movement, isolated physical flicker of sparks, and complex strong light interference models. In other words, the detection results are still not accurate enough. Summary of the Invention

[0005] Therefore, it is necessary to provide an overhead line spark detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the spark detection rate in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for detecting sparks in overhead power lines, including:

[0007] Acquire multimedia data to be processed, which includes at least one multimedia data frame to be processed; each multimedia data frame to be processed includes a structural reference point.

[0008] Each multimedia data frame to be processed is identified to obtain the spark to be confirmed, and the position information of the spark to be confirmed in each multimedia data frame to be processed is obtained. Based on the position information, trajectory data is generated.

[0009] When the trajectory data is in a confirmed state, the first structural nearest neighbor score corresponding to the trajectory data is obtained; the first structural nearest neighbor score is used to characterize the proximity between the trajectory data and the structural reference point.

[0010] Obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and when the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score and the interference counter-evidence score value, generate an alarm message. The interference counter-evidence score value is used to characterize the score value of the interference item that interferes with the spark to be confirmed as the target spark.

[0011] In one embodiment, obtaining the location information of the spark to be confirmed in each multimedia data frame to be processed, and generating trajectory data based on the location information, includes: obtaining a first spark region to be confirmed in the current multimedia data frame to be processed, obtaining a second spark region to be confirmed in the next multimedia data frame to be processed, and determining a spark region association marker between the first spark region to be confirmed and the second spark region to be confirmed; when it is determined that the first spark region to be confirmed and the second spark region to be confirmed can be associated based on the spark region association marker, and the association cost when the first spark region to be confirmed and the second spark region to be confirmed are associated is less than a preset threshold, generating trajectory data based on the first spark region to be confirmed and the second spark region to be confirmed.

[0012] In one embodiment, the overhead line spark detection method includes: when there are other unprocessed multimedia data frames, setting the next multimedia data frame to be processed as the current multimedia data frame to be processed, setting the second spark region to be confirmed as the first spark region to be confirmed, obtaining the next multimedia data frame to be processed of the current multimedia data frame; and continuing to execute the step of obtaining the second spark region to be confirmed in the next multimedia data frame to be processed, until all multimedia data frames to be processed are processed.

[0013] In an optional embodiment, determining the spark region association marker between the first spark region to be confirmed and the second spark region to be confirmed includes: acquiring first coordinate data of a structural reference point in the current multimedia data frame to be processed, and second coordinate data of the structural reference point in the next multimedia data frame to be processed, and determining a global compensation matrix based on the first coordinate data and the second coordinate data; acquiring third coordinate data of the first spark region to be confirmed in the current multimedia data frame to be processed, and determining fourth coordinate data for mapping the first spark region to be confirmed to the next multimedia data frame to be processed based on the third coordinate data and the global compensation matrix; acquiring fifth coordinate data of the second spark region to be confirmed in the next multimedia data frame to be processed, and determining the matching error between the fourth coordinate data and the fifth coordinate data; when the matching error is less than a preset error threshold, setting the spark region association marker between the first spark region to be confirmed and the second spark region to be confirmed as associable.

[0014] In an exemplary embodiment, after obtaining the interference counter-evidence score corresponding to the spark to be confirmed, the overhead line spark detection method further includes: based on the trajectory data, obtaining the brightness feature data of the trajectory data in each multimedia data frame to be processed, and generating a brightness sequence according to the brightness feature data; performing frequency domain analysis on the brightness sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the brightness sequence, and determining the energy ratio according to the first energy and the second energy; obtaining a preset bright-dark alternation template, and determining the correlation coefficient between the brightness sequence and the preset bright-dark alternation template according to the brightness sequence and the preset bright-dark alternation template; generating an alarm message when the spark to be confirmed is determined to be the target spark according to the first structural nearest neighbor score, the interference counter-evidence score, and the energy ratio; or generating an alarm message when the spark to be confirmed is determined to be the target spark according to the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient.

[0015] In one embodiment, obtaining the first structural nearest neighbor score corresponding to the trajectory data includes: for each multimedia data frame to be processed, calculating each second structural nearest neighbor score and each distance value between the spark to be confirmed and the structural reference point; determining the distance variance based on each distance value; and determining the first structural nearest neighbor score based on each second structural nearest neighbor score and the distance variance.

[0016] Secondly, this application also provides an overhead line spark detection device, comprising:

[0017] The data acquisition module is used to acquire multimedia data to be processed, which includes at least one multimedia data frame to be processed; each multimedia data frame to be processed includes a structural reference point.

[0018] The trajectory data generation module is used to identify each multimedia data frame to be processed, obtain the spark to be confirmed, acquire the position information of the spark to be confirmed in each multimedia data frame to be processed, and generate trajectory data based on the position information.

[0019] The structure score acquisition module is used to acquire the first structure nearest neighbor score corresponding to the trajectory data when the trajectory data is in the confirmed state; the first structure nearest neighbor score is used to characterize the proximity between the trajectory data and the structure reference point;

[0020] The alarm information generation module is used to obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and generate alarm prompt information when the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score and the interference counter-evidence score value. The interference counter-evidence score value is used to characterize the score value of the interference item that interferes with the spark to be confirmed as the target spark.

[0021] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method embodiments.

[0022] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0023] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method embodiments.

[0024] The above-mentioned overhead line spark detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product generate trajectory data using the spark to be confirmed. When the trajectory data is in a confirmed state, the first structural nearest neighbor score corresponding to the trajectory data is obtained, and the interference counter-evidence score is obtained. Based on the first structural nearest neighbor score and the interference counter-evidence score, the determination of whether the spark to be confirmed is the target spark is realized. In determining whether the spark to be confirmed is the target spark, the interference counter-evidence score corresponding to the interference item is considered, as well as the relationship between the trajectory data and the structural reference point, to achieve a comprehensive judgment, effectively ensuring the accuracy of the detection results and improving the spark detection rate. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a diagram illustrating the application environment of an overhead line spark detection method in one embodiment.

[0027] Figure 2 This is a flowchart illustrating an overhead line spark detection method in one embodiment;

[0028] Figure 3 This is a schematic diagram of the anchor points of the power equipment structure in one embodiment;

[0029] Figure 4 This is a schematic diagram of the actual spark flicker frequency in one embodiment;

[0030] Figure 5 This is a flowchart illustrating the overhead line spark detection method in another embodiment;

[0031] Figure 6This is a structural block diagram of an overhead line spark detection device in one embodiment;

[0032] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] Overhead lines, distribution lines, and substation external equipment are all crucial components in the power system that transmit electrical energy from power plants to substations or users. Therefore, it is of paramount importance to determine whether there are any safety hazards in overhead lines, distribution lines, and substation external equipment, and to eliminate any such hazards as early as possible.

[0035] When detecting potential safety hazards in overhead lines, distribution lines, and substation equipment, abnormal phenomena such as sparks, discharges, and arcs are typically characterized by short duration, rapid changes in brightness, and localized locations, making them prone to being missed or inaccurate during the detection process.

[0036] In related technologies, target detection models are often used to process inspection videos frame by frame.

[0037] Alternatively, the object detection model can be YOLO, Faster R-CNN, or RT-DETR, etc.

[0038] In one embodiment, each frame of the inspection video can be used as input to the target detection model, and an alarm can be triggered based on the single-frame detection confidence threshold.

[0039] However, strong light interference is commonly seen in overhead line inspection videos, such as lightning, vehicle headlights, solar glare, metallic reflections, and lens glare. These interferences may have similar visual characteristics to sparks in a single frame image, such as height, small area, and local abrupt changes, causing the detection model to easily misidentify them as sparks and generate false alarms. In other words, single-frame detection is prone to inaccurate detection.

[0040] Related technologies also include adding a simple cross-frame association mechanism after the frame-by-frame detection results. For example, it can determine whether the candidate boxes of adjacent frames are consecutive by the intersection-union ratio, the distance between the center points, or the number of consecutive frames, and filter out false alarms in a single frame based on the number of consecutive frames.

[0041] However, relying solely on whether candidate targets appear continuously, whether their positions are stable, or whether the number of consecutive frames exceeds a threshold, it is difficult to effectively distinguish between real discharge sparks and interference such as fixed reflective points, continuous headlights, and lens glare.

[0042] For example, a fixed reflector may remain in a stable position for a long time, headlights may appear continuously for a short period of time, and lens flare may form a continuous trajectory as the camera moves.

[0043] Therefore, simple intersection-union matching, center point distance matching, or continuous frame count judgment are insufficient to meet the requirements for accurate spark detection in complex inspection scenarios.

[0044] Furthermore, overhead line inspection videos are typically captured by drones, robots, or cameras with gimbals. During filming, there are changes in perspective, image shake, gimbal rotation, and changes in target size. If the stability of the spark position is determined directly from the raw pixel coordinates, the spark, which is actually fixed near the wire or fitting, may appear as a drifting image, leading to broken trajectory, missed detection, or misjudgment.

[0045] In one embodiment, real sparks typically occur near electrical equipment structures such as conductors, hardware connection points, insulator ends, clamp edges, and drain wire connection locations, while interference such as lightning, vehicle headlights, and sunlight reflections may occur in the sky, roads, background buildings, or edge areas of the lens. Methods in related technologies focus only on the appearance of the spark candidate frame itself, without determining its spatial relationship with the electrical equipment structure, which easily leads to a large number of false alarms in non-line areas.

[0046] Furthermore, real electrical discharge sparks typically exhibit rapid flickering, alternating bright and dark periods, and brief disappearances. Target detection models may fail to detect targets in darker frames, leading to trajectory interruptions. Directly determining continuity based on the presence of detection boxes would reduce the detection rate of real sparks.

[0047] Therefore, in order to at least solve one of the above-mentioned technical problems, the overhead line spark detection method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 acquires the multimedia data to be processed transmitted by terminal 102. The multimedia data to be processed includes at least one multimedia data frame; each multimedia data frame includes a structural reference point; each multimedia data frame is identified to obtain a spark to be confirmed, and the location information of the spark to be confirmed in each multimedia data frame is acquired, and trajectory data is generated based on the location information; when the trajectory data is in a confirmed state, a first structural nearest neighbor score corresponding to the trajectory data is acquired; the first structural nearest neighbor score is used to characterize the proximity between the trajectory data and the structural reference point; an interference counter-evidence score value corresponding to the spark to be confirmed is acquired, and when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score and the interference counter-evidence score value, an alarm message is generated; the interference counter-evidence score value is used to characterize the score value of interference items that interfere with the spark to be confirmed as the target spark. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0048] In one exemplary embodiment, such as Figure 2 As shown, a method for detecting sparks in overhead power lines is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 208. Wherein:

[0049] Step 202: Obtain the multimedia data to be processed.

[0050] Among them, the multimedia data to be processed is used to characterize the inspection multimedia data to be detected.

[0051] In one embodiment, the multimedia data to be processed can be video stream data or image sequence data.

[0052] Alternatively, multimedia data can be acquired using a multimedia acquisition device.

[0053] For example, multimedia acquisition devices include, but are not limited to, drones, robots, and gimbals.

[0054] In one embodiment, the multimedia data to be processed includes at least one multimedia data frame to be processed.

[0055] In one embodiment, each frame of multimedia data to be processed includes a spark to be confirmed and / or a structural reference point.

[0056] Among them, the unconfirmed spark is used to characterize the unconfirmed area that needs to be confirmed as whether it is the target spark.

[0057] Among them, the structural reference point is used to characterize the reference benchmark point.

[0058] Optionally, the structural reference point can be a fixed point included in the background of the multimedia data frame to be processed, or an anchor point of the power equipment structure.

[0059] For example, the fixed points included in the background of a multimedia data frame may include, but are not limited to, background buildings and roads.

[0060] For example, such as Figure 3 The aforementioned structural anchor points of power equipment may include, but are not limited to, conductor centerlines, insulator areas, hardware areas, and connection point areas.

[0061] In one alternative embodiment, there may be at least one centerline for the conductor.

[0062] In one alternative embodiment, there may be at least one insulator region.

[0063] In one alternative embodiment, the fitting area may be at least one.

[0064] In one alternative embodiment, there may be at least one connection point region.

[0065] In one embodiment, after receiving the multimedia data to be processed, the server can obtain the spark to be confirmed and the structural reference point from the multimedia data to be processed, and perform calculations based on the spark to be confirmed and the structural reference point to determine whether the spark to be confirmed is the target spark.

[0066] In one embodiment, a structural recognition model can be used to identify the multimedia data to be processed in order to determine the structural reference points included in the multimedia data to be processed.

[0067] In one embodiment, the structure recognition model can be a power component detection model, a semantic segmentation model, an instance segmentation model, an edge detection model, or a line detection model.

[0068] In an alternative embodiment, structural reference points can be represented by a set of structures.

[0069] For example, the structural reference point can be represented by the following equation (1):

[0070] (1)

[0071] in, Used to characterize the set of centerlines of a conductor. Used to characterize insulator regions Used to characterize the hardware area Used to characterize the set of connection points.

[0072] Step 204: Identify each multimedia data frame to be processed to obtain the spark to be confirmed, and obtain the position information of the spark to be confirmed in each multimedia data frame to be processed, and generate trajectory data based on the position information.

[0073] In one embodiment, a spark detection model can be pre-trained, the multimedia data to be processed can be input into the spark detection model, and the sparks to be confirmed output by the spark detection model can be obtained.

[0074] In one embodiment, when detecting multimedia data to be processed, the spark detection model can detect each frame of multimedia data to be processed to identify the unconfirmed sparks included in each frame.

[0075] In an optional embodiment, the spark detection model and the structural detection model can be the same. For example, the multimedia data to be processed is input to the spark detection model, and the spark detection model outputs the sparks to be confirmed and structural reference points.

[0076] In an alternative embodiment, the spark detection model and the structure detection model may also be two models that achieve different functions.

[0077] In one embodiment, the spark detection model can be YOLO, Faster R-CNN, RT-DETR, or other object detection models, which are not limited here.

[0078] In an optional embodiment, the spark detection model is a low-threshold model to retain more areas of sparks to be identified, thereby ensuring the spark detection rate.

[0079] Among them, the low-threshold model is used to characterize models with lower confidence thresholds, so as to retain more target objects to be confirmed and ensure the detection rate of target objects.

[0080] In one embodiment, the confidence threshold can be set to 0.2 to 0.4.

[0081] Optionally, the confidence level of the spark to be confirmed output by the spark detection model is greater than or equal to the confidence level threshold.

[0082] In one embodiment, when using a spark detection model to identify each frame of multimedia data to be processed, the resulting frame of multimedia data to be processed may contain one or more sparks to be confirmed.

[0083] Optionally, the unacknowledged sparks corresponding to each multimedia data frame to be processed can be represented in the form of a dataset.

[0084] For example, the unacknowledged spark in frame t can be represented by the following equation (2):

[0085] (2)

[0086] in, The term "spark" is used to characterize the unacknowledged sparks corresponding to the multimedia data frame to be processed in frame t, and "n" is used to characterize the number of unacknowledged sparks included in the multimedia data frame to be processed in frame t. Used to characterize the i-th unconfirmed spark of the multimedia data frame to be processed in frame t.

[0087] In one embodiment, the spark to be confirmed is represented by a parameter vector.

[0088] In one embodiment, the parameter vector may include, but is not limited to, the coordinate parameters, width parameters, height parameters, and confidence level of the region to be confirmed corresponding to the spark to be confirmed.

[0089] The coordinate parameters are used to characterize the coordinates of the region to be confirmed in the multimedia data frame to be processed in frame t. Optionally, the coordinate parameters may include, but are not limited to, the horizontal coordinate parameter and the vertical coordinate parameter.

[0090] The height parameter is used to characterize the height of the area to be confirmed.

[0091] The width parameter is used to characterize the width of the area to be confirmed.

[0092] The confidence level is used to characterize the credibility of the spark detection model's output indicating that the spark to be confirmed is the target spark. Optionally, the confidence level is greater than or equal to a confidence threshold.

[0093] For example, the parameter vector corresponding to the spark to be confirmed can be shown in the following equation (3):

[0094] (3)

[0095] in, The x and y coordinates are used to characterize the i-th unconfirmed spark center of the t-th multimedia data frame to be processed. The width used to characterize the i-th unacknowledged spark of the t-th multimedia data frame to be processed. Used to characterize the height of the i-th unacknowledged spark in the t-th frame of multimedia data to be processed. The confidence level used to characterize the i-th unconfirmed spark of the t-th multimedia data frame to be processed as the target spark.

[0096] In an optional embodiment, after obtaining the sparks to be confirmed included in each multimedia data frame to be processed, the position information of each spark to be confirmed in each multimedia data frame to be processed can be obtained.

[0097] Optionally, the location information can be characterized using coordinate parameters. For example, the coordinates of the i-th unconfirmed spark center point of the t-th multimedia data frame to be processed are: Then the location information can be characterized as .

[0098] In an exemplary embodiment, after confirming the position information of each spark to be confirmed in each multimedia data frame to be processed, trajectory data can be generated based on the position information of each spark to be confirmed in each multimedia data frame to be processed.

[0099] For example, obtaining the location information of the spark to be confirmed in each multimedia data frame to be processed, and generating trajectory data based on the location information, includes: obtaining the first spark region to be confirmed in the current multimedia data frame to be processed, obtaining the second spark region to be confirmed in the next multimedia data frame to be processed, and determining the spark region association marker between the first spark region to be confirmed and the second spark region to be confirmed; when it is determined that the first spark region to be confirmed and the second spark region to be confirmed can be associated based on the spark region association marker, and the association cost when the first spark region to be confirmed and the second spark region to be confirmed are associated is less than a preset threshold, generating trajectory data based on the first spark region to be confirmed and the second spark region to be confirmed.

[0100] In one embodiment, obtaining the first unconfirmed spark region may include: if there are multiple multimedia data frames to be processed, selecting any one of the multimedia data frames to be processed as the current multimedia data frame to be processed, and obtaining the unconfirmed sparks included in the current multimedia data frame to be processed; if there is only one unconfirmed spark included in the current multimedia data frame to be processed, then setting the unconfirmed region corresponding to that unconfirmed spark as the first unconfirmed spark region.

[0101] Optionally, if the current multimedia data frame to be processed includes multiple sparks to be confirmed, a first set of sparks to be confirmed can be generated based on the multiple sparks to be confirmed, one spark to be confirmed can be randomly selected from the first set of sparks to be confirmed, and the randomly selected spark to be confirmed can be set as the first candidate spark, and the area to be confirmed of the first candidate spark in the current multimedia data frame to be processed can be set as the first spark area to be confirmed.

[0102] In one embodiment, a first remaining set of sparks to be confirmed is determined based on a first set of sparks to be confirmed and a first candidate spark, and the first remaining set of sparks to be confirmed is set as the first set of sparks to be confirmed, and the first set of sparks to be confirmed region acquisition steps are executed cyclically.

[0103] Optionally, the first unconfirmed spark region acquisition step may include: acquiring a first candidate spark from the first unconfirmed spark set, setting the unconfirmed region of the first candidate spark in the current multimedia data frame to be processed as the first unconfirmed spark region, and determining a first remaining unconfirmed spark set based on the first unconfirmed spark set and the first candidate spark, and setting the first remaining unconfirmed spark set as the first unconfirmed spark set.

[0104] In one embodiment, obtaining the second unconfirmed spark region in the next unconfirmed multimedia data frame may include: obtaining the next unconfirmed multimedia data frame adjacent to the current unconfirmed multimedia data frame, and obtaining the unconfirmed spark included in the next unconfirmed multimedia data frame. If there is only one unconfirmed spark in the next unconfirmed multimedia data frame, the unconfirmed region of that unconfirmed spark in the next unconfirmed multimedia data frame can be set as the second unconfirmed spark region.

[0105] In one embodiment, if there are multiple unconfirmed sparks in the next multimedia data frame to be processed, a second unconfirmed spark set can be formed based on the unconfirmed sparks included in the next multimedia data frame to be processed.

[0106] Optionally, a second candidate spark is randomly selected from the second set of sparks to be confirmed. It is assumed that the spark is the same spark as the spark to be confirmed or the first candidate spark in the current multimedia data frame to be processed. The region to be confirmed of the second candidate spark in the next multimedia data frame to be processed is obtained, and the region to be confirmed is set as the second spark region to be confirmed.

[0107] In one embodiment, a second remaining set of sparks to be confirmed is generated based on the second set of sparks to be confirmed and the second candidate sparks, and the second remaining set of sparks to be confirmed is set as the second set of sparks to be confirmed. The second set of sparks to be confirmed region acquisition steps are executed cyclically.

[0108] Optionally, the step of obtaining the second unconfirmed spark region may include: selecting one unconfirmed spark from the second unconfirmed spark set and setting the unconfirmed spark as the second candidate spark; setting the unconfirmed region of the second candidate spark in the next multimedia data frame to be processed as the second unconfirmed spark region; generating a second remaining unconfirmed spark set based on the second unconfirmed spark set and the second candidate spark; and setting the second remaining unconfirmed spark set as the second unconfirmed spark set.

[0109] In an exemplary embodiment, after obtaining the first spark region to be confirmed and the second spark region to be confirmed, the spark region association markers corresponding to the first spark region to be confirmed and the second spark region to be confirmed can be used to determine whether trajectory data can be generated based on the first spark region to be confirmed and the second spark region to be confirmed.

[0110] Among them, the spark region association marker is used to characterize the association between the first spark region to be confirmed and the second spark region to be confirmed.

[0111] In one embodiment, the spark region association marker may include, but is not limited to, associated and unassociated.

[0112] For example, when the spark region association is marked as associated, the first spark region to be confirmed and the second spark region to be confirmed can be considered to be associated, and trajectory data can be generated based on the first spark region to be confirmed and the second spark region to be confirmed.

[0113] For example, when the spark region association is marked as not associated, it is considered that the first spark region to be confirmed and the second spark region to be confirmed are not associated, and trajectory data cannot be generated based on the first spark region to be confirmed and the second spark region to be confirmed.

[0114] In one exemplary embodiment, the step of determining the spark region association marker may include:

[0115] The system acquires the first coordinate data of the structural reference point in the current multimedia data frame to be processed, and the second coordinate data of the second structural point in the next multimedia data frame to be processed. A global compensation matrix is ​​then determined based on the first and second coordinate data. The system acquires the third coordinate data of the first spark region to be confirmed in the current multimedia data frame to be processed, and determines the fourth coordinate data for mapping the first spark region to be confirmed to the next multimedia data frame to be processed based on the third coordinate data and the global compensation matrix. The system acquires the fifth coordinate data of the second spark region to be confirmed in the next multimedia data frame to be processed, and determines the matching error between the fourth and fifth coordinate data. When the matching error is less than a preset error threshold, the spark region association marker between the first and second spark regions to be confirmed is set as associable.

[0116] In one exemplary embodiment, it is assumed that the structural reference point is in the current multimedia data frame to be processed. The first coordinate data in is In the next multimedia data frame to be processed The second coordinate data in is .

[0117] Optionally, it can be based on the first coordinate data Second coordinate data Determine the global compensation matrix between the current multimedia data frame to be processed and the next multimedia data frame to be processed. .

[0118] In one embodiment, it can also be assumed that the first spark region to be confirmed is mapped to the next multimedia data frame to be processed, and the coordinate data when mapped to the next multimedia data frame to be processed is obtained.

[0119] For example, suppose the first spark region to be confirmed is in the current multimedia data frame to be processed. The coordinate data is the third coordinate data, such as... It can be based on the third coordinate data and the global compensation matrix The hypothesis is that the first unconfirmed spark region will be mapped to the next unprocessed multimedia data frame. Yes, the fourth coordinate data is... .

[0120] In one embodiment, the matching error can be determined based on the actual coordinate data of the spark to be confirmed in the next multimedia data frame to be processed and the fourth coordinate data, and the spark region association mark can be determined based on the matching error.

[0121] For example, the fifth coordinate data corresponding to the second spark region to be confirmed in the next multimedia data frame to be processed is obtained, the matching error between the fourth coordinate data and the fifth coordinate data is determined, and the key marker of the spark region is determined based on the matching error.

[0122] For example, matching error It can be shown in the following formula (4):

[0123] (4)

[0124] in, Used to characterize the fourth coordinate data, Used to characterize the fifth coordinate data, Used to determine the matching error between the fourth and fifth coordinate data.

[0125] In an optional embodiment, if the matching error is less than or equal to a preset matching error threshold, it indicates that the first spark region to be confirmed and the second spark region to be confirmed are stable relative to the power equipment structure. It can be considered that the first spark region to be confirmed and the second spark region to be confirmed can be associated to form trajectory data. At this time, the spark region association mark is set as associative.

[0126] In an optional embodiment, if the matching error is greater than a preset matching error threshold, it indicates that there is a significant drift between the first spark to be confirmed and the second spark to be confirmed. This may be due to interference from headlights, birds, lens glare, or moving reflections. The first spark to be confirmed and the second spark to be confirmed cannot be associated to form trajectory data. In this case, the spark area association marker is set to not associated.

[0127] In an exemplary embodiment, when it is determined that the first spark region to be confirmed and the second spark region to be confirmed can be associated based on the spark region association marker, the association cost when the first spark region to be confirmed and the second spark region to be confirmed are also confirmed. If the association cost is less than a preset association threshold, trajectory data is generated based on the first spark region to be confirmed and the second spark region to be confirmed.

[0128] The association cost is used to determine whether the first and second spark regions to be confirmed can be considered the same spark.

[0129] In one embodiment, the association cost can be determined based on the candidate box intersection-union ratio, center point distance, structural anchor point distance variation, and motion compensation residual.

[0130] The candidate box intersection-union ratio is used to characterize the degree of overlap between the candidate boxes corresponding to the first and second unconfirmed spark regions.

[0131] For example, a first area of ​​the candidate box corresponding to the first spark region to be confirmed and a second area of ​​the candidate box corresponding to the second spark region to be confirmed can be determined, and the intersection-union ratio of the candidate boxes can be determined based on the first area and the second area.

[0132] The center point distance is used to characterize the distance between the first center point of the candidate box corresponding to the first spark region to be confirmed and the second center point of the candidate box corresponding to the second spark region to be confirmed.

[0133] Optionally, the center point distance can be the Euclidean distance between the first center point and the second center point, or it can be the normalized distance.

[0134] Among them, the structural anchor point distance change is used to describe the change in the distance between the spark to be confirmed and the structural reference point in two adjacent frames.

[0135] Optionally, the first distance between the first spark region to be confirmed and the structural reference point support can be obtained in the current multimedia data frame to be processed, and the second distance between the second spark region to be confirmed and the structural reference point support can be obtained in the next multimedia data frame to be processed. The distance change is determined based on the first distance and the second distance, and the distance change is the structural anchor point distance change.

[0136] Among them, motion compensation residual is used to characterize the "residual deviation" that still exists between the predicted position of the target in the previous frame and the actual observed position in the current frame after global motion compensation alignment.

[0137] For example, the coordinate data of the structural reference point in the current multimedia data frame to be processed can be obtained. Based on the coordinate data and the global compensation matrix, the coordinate data of the assumed structural reference point when mapped to the next multimedia data to be processed can be determined. The coordinate data of the structural reference point in the next multimedia data to be processed can be obtained. Based on the coordinate data of the structural reference point in the next multimedia data to be processed and the coordinate data of the assumed structural reference point when mapped to the next multimedia data to be processed, the motion compensation residual can be determined.

[0138] For example, the formula for calculating the associated cost is shown in equation (5) below:

[0139] (5)

[0140] in, Used to characterize the cost of association Used to characterize the intersection-union ratio of candidate boxes Used to characterize the distance to the center point Used to characterize changes in the distance between structural anchor points Used to characterize motion compensation residuals , , and Used to characterize weighting coefficients.

[0141] In one embodiment, it can also be determined whether there are other unprocessed multimedia data frames. If there are other unprocessed multimedia data frames, the next multimedia data frame to be processed can be set as the current multimedia data frame to be processed, the second spark region to be confirmed can be set as the first spark region to be confirmed, the next multimedia data frame to be processed of the current multimedia data frame to be processed can be obtained, and the step of obtaining the second spark region to be confirmed in the next multimedia data frame to be processed can be continued until all multimedia data frames to be processed are processed.

[0142] In one embodiment, the trajectory data can be generated based on the region to be confirmed in each frame of multimedia data to be processed, where the region to be confirmed in each frame can be characterized by coordinate parameters, and the trajectory data can be characterized by a set of coordinate parameters for the region to be confirmed in each frame.

[0143] For example, the trajectory data can be described by the following formula (6):

[0144] (6)

[0145] in, Used to characterize the k-th trajectory data, The region to be confirmed in the i-th frame is represented by m, and m represents the total number of regions to be confirmed.

[0146] Step 206: When the trajectory data is in the confirmed state, obtain the first structural nearest neighbor score corresponding to the trajectory data.

[0147] Understandably, there may be one or more unconfirmed sparks in the current multimedia data frame to be processed, and the corresponding first unconfirmed spark region may be one or more. The unconfirmed sparks in the next multimedia data frame to be processed may also be one or more, and the corresponding second unconfirmed spark region may be one or more. Therefore, the trajectory data generated based on the first unconfirmed spark region and the second unconfirmed spark may be one or more. In order to ensure the accuracy of spark detection based on trajectory data, it can be determined whether spark detection can be achieved based on trajectory data based on the state data of the trajectory data.

[0148] Among them, the state data is used to characterize the current state of the trajectory data, and the state data can be used to determine whether the trajectory data can be used to realize spark detection.

[0149] For example, status data includes, but is not limited to, acknowledged status, rejected status, and keep-track status.

[0150] Among them, the confirmed state is used to characterize the trajectory data as being in a usable state, and the spark detection processing can be carried out based on the trajectory data.

[0151] Among them, the rejection state is used to characterize that the trajectory data cannot be used to realize spark detection. The trajectory data may be generated by interference features such as lightning, car lights, reflections or glare, and the trajectory data should be discarded.

[0152] Among them, the tracking state is used to characterize situations where there is insufficient evidence for the current trajectory data, and further observation is required.

[0153] In one embodiment, state data can be determined based on the accumulated integral value of evidence.

[0154] For example, the following can be obtained: power frequency flicker fingerprint matching degree, structural anchoring consistency score, motion compensation stability score, spark color consistency score, and interference rebuttal score. The following weights can also be obtained: the first weight corresponding to the power frequency flicker fingerprint matching degree, the second weight corresponding to the structural anchoring consistency score, the third weight corresponding to the motion compensation stability score, the fourth weight corresponding to the spark color consistency score, and the fifth weight corresponding to the interference rebuttal score. Based on the obtained power frequency flicker fingerprint matching degree, structural anchoring consistency score, motion compensation stability score, spark color consistency score, interference rebuttal score, and the first, second, third, fourth, and fifth weights, the cumulative evidence score can be determined.

[0155] Among them, the power frequency scintillation fingerprint matching degree is used to characterize the matching procedure between the scintillation frequency of the spark to be confirmed and the scintillation frequency of the target spark, and is used to determine whether the spark to be confirmed is indeed the target spark.

[0156] Optionally, the power frequency scintillation fingerprint matching degree can also be used to determine that the sparks to be confirmed in two adjacent frames are the same spark to be confirmed.

[0157] Among them, the structural anchoring consistency score is used to characterize the relative geometric relationship between structural reference points and each multimedia data frame to be processed.

[0158] Optionally, the structural anchoring consistency score can be determined by the first structural neighbor score. For example, the structural anchoring consistency score is equal to the first structural neighbor score.

[0159] Among them, the motion compensation stability score represents the spatial stability of the trajectory data after motion compensation.

[0160] For example, the matching error between adjacent frames of the same trajectory data is obtained, and the motion compensation stability score is obtained by summing the matching errors.

[0161] Among them, the spark color consistency score is used to characterize the matching procedure between the area to be confirmed corresponding to the spark to be confirmed and the typical color of the spark.

[0162] For example, the proportion of a preset color in the spark color pixels within the area to be confirmed for the spark to be confirmed, as well as the stability of color changes, can be obtained. Based on the proportion and the stability of color changes, a spark color consistency score can be determined. Optionally, the preset color may include, but is not limited to, orange, yellow, and blue-white.

[0163] Among them, the interference counter-evidence score is used to characterize the score of interference items that interfere with the spark to be confirmed as the target spark.

[0164] Optionally, interference items include, but are not limited to, vehicle lights, lightning, fixed reflections, and lens glare.

[0165] In one embodiment, the interference rebuttal score value The calculation formula can be shown in equation (7) below:

[0166] (7)

[0167] in, , , , denoted by , respectively, the counter-evidence scores for lightning, headlights, fixed reflections, and lens flare, with a, b, c, and d being weighting parameters.

[0168] Optionally, It can obtain the area ratio of the local highlighted region in the current multimedia data frame to be processed, and determine the region based on the area ratio of the local highlighted region.

[0169] Optionally, The recognition model can be used to identify vehicles in the current multimedia data frame to be processed, obtain the area of ​​the detection box containing the vehicle and the corresponding confidence level, and determine the vehicle based on the area of ​​the detection box containing the vehicle and the confidence level.

[0170] Optionally, It can identify the mirror region symmetrical to the spark to be confirmed in the current multimedia data frame to be processed, and obtain the brightness contrast of the mirror region, and determine the result based on the brightness contrast.

[0171] Optionally, The skewness of the global luminance histogram of the current multimedia data frame to be processed can be obtained, and the determination can be made based on the skewness of the global luminance histogram.

[0172] For example, the cumulative evidence score The formula for determining can be shown in equation (8) below:

[0173] (8)

[0174] in, Used to represent the cumulative integral value of the current frame. For historical evidence amnesia factor, This is the accumulated integral value from the previous frame. For power frequency flashing fingerprint matching accuracy, Assign a consistency score to the structure's anchoring. For motion compensation stability score, The score is for the consistency of spark color. To interfere with the score of the counter-evidence, As the first weight, As the second weight, As the third weight, As the fourth weight, It is the fifth weight.

[0175] In one embodiment, after determining the cumulative evidence integral value, the state data of the trajectory data can be determined based on the cumulative evidence integral value.

[0176] Optionally, a preset state threshold can be obtained, and the state data corresponding to the trajectory data can be determined based on the relationship between the accumulated evidence integral value and the preset state threshold.

[0177] In one embodiment, the preset status threshold includes, but is not limited to, the confirmation status threshold and the rejection status threshold.

[0178] For example, if If so, the trajectory data can be considered to be in a confirmed state.

[0179] For example, if If the trajectory data is rejected, then it is considered to be in a rejection state.

[0180] For example, if If so, the trajectory data is considered to be in a tracking state.

[0181] in, To confirm the state threshold, the minimum threshold data used to determine the state of the trajectory data as confirmed is used. The rejection state threshold is the highest threshold value used to characterize the state data of the trajectory data that is in a rejection state.

[0182] Optionally, .

[0183] Among them, the first structural nearest neighbor score is used to characterize the proximity between the trajectory data and the structural reference point.

[0184] In one embodiment, when the trajectory data is determined to be in an acknowledged state, the first structural nearest neighbor score corresponding to the trajectory data can be determined.

[0185] In an exemplary embodiment, the first structural nearest neighbor score can be determined based on the second structural nearest neighbor score. For example: for each multimedia data frame to be processed, calculate each second structural nearest neighbor score and each distance value between the spark to be confirmed and the structural reference point; determine the distance variance based on each distance value; and determine the first structural nearest neighbor score based on each second structural nearest neighbor score and the distance variance.

[0186] The second structural nearest neighbor score is used to characterize the proximity between the spark to be confirmed and the structural reference point in each multimedia data frame to be processed.

[0187] For example, suppose the structural reference point in the multimedia data frame to be processed in the current frame can be .

[0188] in, Represents the set of centerlines of a conductor. Indicates the insulator region. Represents the hardware area, This represents the set of connection points.

[0189] Optionally, for the spark to be confirmed, the minimum distance between the spark to be confirmed and the structural reference point can be determined. As shown in equation (9):

[0190] (9)

[0191] In one embodiment, after determining the minimum distance, the nearest neighbor score of the second structure can be determined based on the minimum distance.

[0192] For example, the second structural nearest neighbor score The determination of is shown in the following formula (11):

[0193] (11)

[0194] in, Used to characterize distance decay parameters.

[0195] Optionally, after determining the nearest neighbor score of the second structure, the nearest neighbor score of the first structure can be determined based on the nearest neighbor score of the second structure.

[0196] For example, the first structural nearest neighbor score The determination of can be shown in the following formula (12):

[0197] (12)

[0198] in, Used to characterize the distance fluctuation penalty coefficient This represents the variance of the distance from the candidate target to the nearest power structure anchor point.

[0199] Step 208: Obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and generate an alarm message when the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score and the interference counter-evidence score value.

[0200] In one embodiment, the confirmation of whether the spark to be confirmed is the target spark can be achieved based on the first structural nearest neighbor score and the interference reverse score.

[0201] For example, if it is determined that the spark to be confirmed is the target spark, an alarm message can be generated.

[0202] The alarm information is used to indicate the presence of sparks in the overhead lines.

[0203] Optionally, alarm information can be sent from the server to the terminal so that users can query the corresponding alarm information when using the terminal.

[0204] In one embodiment, when the server sends alarm information to the terminal, it can also set the prompt format for the alarm information and display the alarm information on the terminal in a prompt format.

[0205] Understandably, the flickering of real sparks exhibits certain regularities, such as... Figure 4 Therefore, the power frequency flashing fingerprint can be used to confirm whether the spark to be confirmed is the target spark.

[0206] In an exemplary embodiment, the trajectory data can be used as a basis to obtain luminance feature data of the trajectory data in each multimedia data frame to be processed, and a luminance sequence can be generated based on the luminance feature data; frequency domain analysis can be performed on the luminance sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the luminance sequence, and the energy ratio can be determined based on the first energy and the second energy; a preset brightness alternation template can be obtained, and the correlation coefficient between the luminance sequence and the preset brightness alternation template can be determined based on the luminance sequence and the preset brightness alternation template; if the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the energy ratio, an alarm message can be generated; or, if the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient, an alarm message can be generated.

[0207] In one embodiment, each trajectory point involved in the trajectory data can be obtained, and the brightness data of each trajectory point in the corresponding multimedia data frame to be processed can be determined. A brightness sequence can be generated based on the brightness data corresponding to each trajectory point.

[0208] For example, brightness sequence It can be shown in the following formula (13):

[0209] (13)

[0210] in, The i-th trajectory point is used to represent the total number of trajectory points (N).

[0211] In one embodiment, multi-dimensional feature data corresponding to the region to be confirmed in each multimedia data frame to be processed can be determined based on each trajectory point, and multi-dimensional time-series signals can be generated based on the multi-feature data.

[0212] For example, multidimensional time-series signals It can be shown in the following formula (14):

[0213] (14)

[0214] in, For the average brightness of the candidate region, For the maximum brightness of the candidate region, The proportion of orange-yellow or blue-white spark colors, For candidate box area, To test confidence level, The distance between the candidate box and the structural anchor point, To compensate for motion residuals.

[0215] In one embodiment, the trajectory point can be the center point of the region to be confirmed in each multimedia data frame to be processed, or any other structural point, as long as it can be used to represent the region to be confirmed, and there is no limitation thereto.

[0216] In one embodiment, frequency domain analysis can be performed on the luminance sequence to obtain the energy of different frequency components. .

[0217] Optionally, after obtaining the energy of different frequency band components, the energy proportion of the power frequency related frequency band can be calculated.

[0218] For example, the energy proportion of power frequency related frequency bands The calculation formula can be shown in the following formula (15):

[0219] (15)

[0220] in, The target frequency band is related to the aliasing of power grid frequency and its video sampling. This refers to the total analysis frequency band.

[0221] Optionally, if If the frequency band proportion exceeds the set threshold, the trajectory data can be considered to conform to the periodic flickering pattern of a real spark, thus allowing for the utilization of... Further determine whether the spark corresponding to the trajectory data is indeed the target spark.

[0222] In an exemplary embodiment, if the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score, the interference counter-evidence score, and the energy ratio, then the spark to be confirmed can be considered the target spark, and an alarm message is generated. In the above embodiment, when determining whether the spark to be confirmed is the target spark, power frequency flicker fingerprint data is considered, which can ensure the accuracy of spark detection and improve the spark detection rate.

[0223] In an optional embodiment, a preset brightness alternation template can also be obtained, and the correlation coefficient between the brightness sequence and the preset brightness alternation template can be determined based on the preset brightness alternation template.

[0224] Among them, the preset alternating brightness template is used to represent the reference sequence of the actual spark brightness alternating between increasing and decreasing over time.

[0225] Optionally, the correlation coefficient The calculation formula can be shown in the following equation (16):

[0226] (16)

[0227] Where M is used to characterize the preset alternating bright and dark template, and corr() is used to determine the correlation coefficient.

[0228] In one exemplary embodiment, if If the correlation coefficient is greater than a preset threshold, the trajectory data can be considered to conform to the periodic flashing pattern of a real spark, and thus can be used... Further determine whether the spark corresponding to the trajectory data is indeed the target spark.

[0229] For example, when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient, an alarm message is generated.

[0230] In one optional embodiment, the alarm information may include, but is not limited to, the time of the spark occurrence, the location of the spark, the duration of the spark, the spark confidence level, the associated power structure type, the trajectory log, and keyframe screenshots.

[0231] In one embodiment, when acquiring each trajectory point involved in the trajectory data, determining the brightness data of each trajectory point in the corresponding multimedia data frame to be processed, and generating a brightness sequence based on the brightness data corresponding to each trajectory point, the trajectory data can also be filled with dark frame hidden state.

[0232] Optionally, the trajectory data may be checked for missing frames. If missing frames are found, the trajectory data may be padded with frames.

[0233] For example, assuming that there are unconfirmed sparks in the trajectory data in both frame t-1 and frame t+1, but no unconfirmed sparks in frame t, then frame t is padded with frames t-1 and t+1.

[0234] Optionally, obtain the region to be confirmed corresponding to the spark to be confirmed in frame t-1. And the region to be confirmed corresponding to the spark to be confirmed in frame t+1. According to the region to be confirmed in frame t-1 The region to be confirmed corresponding to the spark to be confirmed in frame t+1. Determine the region to be confirmed corresponding to frame t.

[0235] For example, the region to be determined corresponding to the t-th frame The calculation formula can be shown in the following equation (17):

[0236] (17)

[0237] In one embodiment, the data frame after frame interpolation is processed to determine whether the interpolated frame is a dark frame.

[0238] Optionally, processing the data frame after frame interpolation may include, but is not limited to, the region to be determined corresponding to the t-th frame. Internal calculation of local weak spark response.

[0239] In one embodiment, if the local weak spark response meets the preset requirements, the t-th frame data frame can be considered a hidden spark frame, and the t-th frame can be marked as a dark frame.

[0240] Optionally, weak spark response includes, but is not limited to, local brightness residual, spark color residual, edge response, distance from structural anchor point, and consistency with the brightness change trend of previous and subsequent frames.

[0241] Among them, the local brightness residual is used to characterize the (t-1)th frame. With frame t The local brightness difference between them.

[0242] Among them, the spark color residual is used to characterize the region to be confirmed in the t-th frame. The color difference between the background image included in the multimedia data to be processed in frame t.

[0243] The edge response is used to characterize the edge features of the region to be confirmed in the t-th frame image. It is the gradient magnitude or direction response generated after filtering by a convolution kernel (such as Sobel, Canny, or LoG).

[0244] Among them, the consistency of brightness change trends between consecutive frames is used to characterize the direction, magnitude and acceleration of brightness change in the region (or the whole image) corresponding to the spark to be confirmed between adjacent frames, and whether it remains smooth, monotonous or conforms to specific physical laws in the time series.

[0245] In the above-mentioned overhead line spark detection method, trajectory data is generated using the spark to be confirmed. While the trajectory data is in a confirmed state, the first structural nearest neighbor score and the interference counter-evidence score corresponding to the trajectory data are obtained. Based on the first structural nearest neighbor score and the interference counter-evidence score, it is determined whether the spark to be confirmed is the target spark. When confirming whether the spark to be confirmed is the target spark, the interference counter-evidence score corresponding to the interference item and the correspondence between the spark to be confirmed and the structural reference point are considered, achieving a comprehensive judgment and effectively ensuring the accuracy of the detection results and improving the spark detection rate. Furthermore, using a low-threshold model to read as many sparks to be confirmed as possible from the multimedia data can ensure the quantity of sparks to be confirmed, thereby ensuring the spark detection detection rate. Moreover, factors such as power frequency flicker fingerprints and color proportions can also be considered to effectively ensure the accuracy of spark detection and improve the spark detection rate when confirming whether the spark to be confirmed is the target spark.

[0246] In one exemplary embodiment, such as Figure 5 As shown, a method for detecting sparks in overhead power lines is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 502 to 536, wherein:

[0247] Step 502: Obtain the multimedia data to be processed.

[0248] Optionally, the multimedia data to be processed includes at least one multimedia data frame to be processed.

[0249] In one embodiment, each multimedia data frame to be processed includes a structural reference point.

[0250] Step 504: Identify each multimedia data frame to be processed to obtain the spark to be confirmed.

[0251] Step 506: Obtain the first unconfirmed spark region in the current multimedia data frame to be processed.

[0252] Step 508: Obtain the second spark region to be confirmed in the next multimedia data frame to be processed.

[0253] Step 510: Obtain the first coordinate data of the structural reference point in the current multimedia data frame to be processed, and the second coordinate data of the structural reference point in the next multimedia data frame to be processed, and determine the global compensation matrix based on the first coordinate data and the second coordinate data.

[0254] Step 512: Obtain the third coordinate data of the first spark region to be confirmed in the current multimedia data frame to be processed, and determine the fourth coordinate data when mapping the first spark region to be confirmed to the next multimedia data frame to be processed based on the third coordinate data and the global compensation matrix.

[0255] Step 514: Obtain the fifth coordinate data of the second spark region to be confirmed in the next multimedia data frame to be processed, and determine the matching error between the fourth coordinate data and the fifth coordinate data.

[0256] Step 516: When the matching error is less than the preset error threshold, set the spark region association mark between the first spark region to be confirmed and the second spark region to be confirmed as associatable.

[0257] Step 518: When it is determined that the first spark region to be confirmed and the second spark region to be confirmed can be associated according to the spark region association marker, and the association cost when the first spark region to be confirmed and the second spark region to be confirmed are associated is less than a preset threshold, trajectory data is generated based on the first spark region to be confirmed and the second spark region to be confirmed.

[0258] Step 520: Determine if there are any other unprocessed multimedia data frames. If yes, proceed to step 522; otherwise, proceed to step 524.

[0259] Step 522: Set the next multimedia data frame to be processed as the current multimedia data frame to be processed, set the second spark area to be confirmed as the first spark area to be confirmed, obtain the next multimedia data frame to be processed of the current multimedia data frame to be processed, and continue to execute step 508.

[0260] Step 524: When the trajectory data is in the confirmed state, calculate the second structural nearest neighbor scores and distance values ​​between the spark to be confirmed and the structural reference point for each multimedia data frame to be processed.

[0261] Among them, the second structural neighbor molecules are used to characterize the proximity between the spark to be identified and the structural reference point.

[0262] Step 526: Determine the distance variance based on each distance value, and determine the first structure nearest neighbor score based on each second structure nearest neighbor score and distance variance.

[0263] Among them, the first structural nearest neighbor score is used to characterize the proximity between the trajectory data and the structural reference point.

[0264] Step 528: Obtain the interference counter-evidence score value corresponding to the spark to be confirmed.

[0265] Step 530: Based on the trajectory data, obtain the brightness feature data of the trajectory data in each multimedia data frame to be processed, and generate a brightness sequence based on the brightness feature data.

[0266] Step 532: Perform frequency domain analysis on the brightness sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the brightness sequence, and determine the energy ratio based on the first energy and the second energy.

[0267] Step 534: Obtain the preset brightness alternation template, and determine the correlation coefficient between the brightness sequence and the preset brightness alternation template based on the brightness sequence and the preset brightness alternation template.

[0268] Step 536: If the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score, the interference counter-evidence score, and the energy ratio, or if the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score, the interference counter-evidence score, and the correlation coefficient, an alarm message is generated.

[0269] In the above-mentioned overhead line spark detection method, trajectory data is generated using the spark to be confirmed. While the trajectory data is in a confirmed state, the first structural nearest neighbor score and the interference counter-evidence score corresponding to the trajectory data are obtained. Based on the first structural nearest neighbor score and the interference counter-evidence score, it is determined whether the spark to be confirmed is the target spark. When confirming whether the spark to be confirmed is the target spark, the interference counter-evidence score corresponding to the interference item and the correspondence between the spark to be confirmed and the structural reference point are considered, achieving a comprehensive judgment and effectively ensuring the accuracy of the detection results and improving the spark detection rate. Furthermore, using a low-threshold model to read as many sparks to be confirmed as possible from the multimedia data can ensure the quantity of sparks to be confirmed, thereby ensuring the spark detection detection rate. Moreover, factors such as power frequency flicker fingerprints and color proportions can also be considered to effectively ensure the accuracy of spark detection and improve the spark detection rate when confirming whether the spark to be confirmed is the target spark.

[0270] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0271] Based on the same inventive concept, this application also provides an overhead line spark detection device for implementing the overhead line spark detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more overhead line spark detection device embodiments provided below can be found in the limitations of the overhead line spark detection method described above, and will not be repeated here.

[0272] In one exemplary embodiment, such as Figure 6 As shown, an overhead line spark detection device 600 is provided, including: a data acquisition module 602, a trajectory data generation module 604, a structural score acquisition module 606, and an alarm information generation module 608, wherein:

[0273] The data acquisition module 602 is used to acquire multimedia data to be processed, which includes at least one multimedia data frame to be processed; each multimedia data frame to be processed includes a structural reference point.

[0274] The trajectory data generation module 604 is used to identify each multimedia data frame to be processed, obtain the spark to be confirmed, acquire the position information of the spark to be confirmed in each multimedia data frame to be processed, and generate trajectory data based on the position information.

[0275] The structure score acquisition module 606 is used to acquire the first structure nearest neighbor score corresponding to the trajectory data when the trajectory data is in the confirmed state; the first structure nearest neighbor score is used to characterize the proximity between the trajectory data and the structure reference point.

[0276] The alarm information generation module 608 is used to obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and generate alarm prompt information when the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score and the interference counter-evidence score value. The interference counter-evidence score value is used to characterize the score value of the interference item that interferes with the spark to be confirmed as the target spark.

[0277] In one embodiment, the trajectory data generation module is further configured to acquire a first unconfirmed spark region in the current multimedia data frame to be processed, acquire a second unconfirmed spark region in the next multimedia data frame to be processed, and determine a spark region association marker between the first unconfirmed spark region and the second unconfirmed spark region; when it is determined that the first unconfirmed spark region and the second unconfirmed spark region can be associated according to the spark region association marker, and the association cost when the first unconfirmed spark region and the second unconfirmed spark region are associated is less than a preset threshold, trajectory data is generated based on the first unconfirmed spark region and the second unconfirmed spark region.

[0278] In an optional embodiment, the overhead line spark detection device further includes a loop module, used to, when there are still other unprocessed multimedia data frames, set the next multimedia data frame to be processed as the current multimedia data frame to be processed, set the second spark region to be confirmed as the first spark region to be confirmed, obtain the next multimedia data frame to be processed of the current multimedia data frame to be processed; and continue to execute the step of obtaining the second spark region to be confirmed in the next multimedia data frame to be processed, until all multimedia data frames to be processed are processed.

[0279] In an exemplary embodiment, the trajectory data generation module is further configured to: acquire first coordinate data of the structural reference point in the current multimedia data frame to be processed, and second coordinate data of the structural reference point in the next multimedia data frame to be processed; determine a global compensation matrix based on the first and second coordinate data; acquire third coordinate data of the first spark region to be confirmed in the current multimedia data frame to be processed, and determine fourth coordinate data for mapping the first spark region to be confirmed to the next multimedia data frame to be processed based on the third coordinate data and the global compensation matrix; acquire fifth coordinate data of the second spark region to be confirmed in the next multimedia data frame to be processed, and determine the matching error between the fourth and fifth coordinate data; and when the matching error is less than a preset error threshold, set the spark region association mark between the first and second spark regions to be confirmed as associatable.

[0280] In one embodiment, the alarm information generation module is used to acquire brightness feature data of the trajectory data in each multimedia data frame to be processed based on the trajectory data, and generate a brightness sequence based on the brightness feature data; perform frequency domain analysis on the brightness sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the brightness sequence, and determine the energy ratio based on the first energy and the second energy; acquire a preset brightness alternation template, and determine the correlation coefficient between the brightness sequence and the preset brightness alternation template based on the brightness sequence and the preset brightness alternation template; generate an alarm prompt message when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the energy ratio; or, generate an alarm prompt message when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient.

[0281] In one embodiment, the structure score acquisition module is used to calculate, for each multimedia data frame to be processed, the second structure nearest neighbor scores and the distance values ​​between the spark to be confirmed and the structure reference point; determine the distance variance based on the distance values; and determine the first structure nearest neighbor score based on the second structure nearest neighbor scores and the distance variance.

[0282] Each module in the aforementioned overhead line spark detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0283] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 The computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for detecting sparks on overhead power lines.

[0284] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0285] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0286] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0287] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0288] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0289] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0290] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0291] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting sparks on overhead power lines, characterized in that, The method includes: Acquire multimedia data to be processed, the multimedia data to be processed including at least one multimedia data frame to be processed; each multimedia data frame to be processed includes a structural reference point. Each of the multimedia data frames to be processed is identified to obtain the spark to be confirmed, and the first spark region to be confirmed in the current multimedia data frame is obtained, the second spark region to be confirmed in the next multimedia data frame is obtained, and the first coordinate data of the structural reference point in the current multimedia data frame and the second coordinate data of the structural reference point in the next multimedia data frame are obtained, and a global compensation matrix is ​​determined based on the first coordinate data and the second coordinate data. Obtain the third coordinate data of the first spark region to be confirmed in the current multimedia data frame to be processed, and determine the fourth coordinate data when mapping the first spark region to be confirmed to the next multimedia data frame to be processed based on the third coordinate data and the global compensation matrix. Obtain the fifth coordinate data of the second unconfirmed spark region in the next multimedia data frame to be processed, and determine the matching error between the fourth coordinate data and the fifth coordinate data; When the matching error is less than a preset error threshold, the spark region association mark between the first spark region to be confirmed and the second spark region to be confirmed is set as associative. When it is determined that the first spark region to be confirmed and the second spark region to be confirmed can be associated according to the spark region association marker, and the association cost when the first spark region to be confirmed and the second spark region to be confirmed are associated is less than a preset threshold, trajectory data is generated based on the first spark region to be confirmed and the second spark region to be confirmed. When the trajectory data is in a confirmed state, a first structural nearest neighbor score corresponding to the trajectory data is obtained; the first structural nearest neighbor score is used to characterize the proximity between the trajectory data and the structural reference point. Obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score and the interference counter-evidence score value, generate an alarm message. The interference counter-evidence score value is used to characterize the score value of the interference item that interferes with the spark to be confirmed as the target spark.

2. The method according to claim 1, characterized in that, The method includes: If there are other unprocessed multimedia data frames, the next multimedia data frame to be processed is set as the current multimedia data frame to be processed, the second unconfirmed spark region is set as the first unconfirmed spark region, and the next multimedia data frame to be processed of the current multimedia data frame is obtained; and the step of obtaining the second unconfirmed spark region in the next multimedia data frame to be processed is continued until all multimedia data frames to be processed are processed.

3. The method according to claim 1, characterized in that, After obtaining the interference counter-evidence score value corresponding to the spark to be confirmed, the method includes: Based on the trajectory data, the brightness feature data of the trajectory data in each of the multimedia data frames to be processed is obtained, and a brightness sequence is generated according to the brightness feature data; Frequency domain analysis is performed on the brightness sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the brightness sequence, and the energy ratio is determined based on the first energy and the second energy. Obtain a preset brightness alternation template, and determine the correlation coefficient between the brightness sequence and the preset brightness alternation template based on the brightness sequence and the preset brightness alternation template; If the spark to be confirmed is determined to be the target spark based on the first structure nearest neighbor score, the interference counter-evidence score, and the energy ratio, an alarm message is generated. Alternatively, if the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient, an alarm message is generated.

4. The method according to claim 1, characterized in that, The step of obtaining the first structural nearest neighbor score corresponding to the trajectory data includes: For each of the multimedia data frames to be processed, calculate the second structural nearest neighbor scores and distance values ​​between the spark to be confirmed and the structural reference point; The distance variance is determined based on each of the distance values, and the first structure nearest neighbor score is determined based on each of the second structure nearest neighbor scores and the distance variance.

5. A spark detection device for overhead power lines, characterized in that, The device includes: The data acquisition module is used to acquire multimedia data to be processed, the multimedia data to be processed including at least one multimedia data frame to be processed; each multimedia data frame to be processed includes a structural reference point. The trajectory data generation module is used to identify each of the multimedia data frames to be processed, obtain the spark to be confirmed, and acquire the first spark region to be confirmed in the current multimedia data frame, acquire the second spark region to be confirmed in the next multimedia data frame, acquire the first coordinate data of the structural reference point in the current multimedia data frame, and the second coordinate data of the structural reference point in the next multimedia data frame, and determine a global compensation matrix based on the first and second coordinate data; acquire the third coordinate data of the first spark region to be confirmed in the current multimedia data frame, and determine the first spark region to be confirmed based on the third coordinate data and the global compensation matrix. The fourth coordinate data of the spark region when it is mapped to the next multimedia data frame to be processed; the fifth coordinate data of the second spark region to be confirmed in the next multimedia data frame to be processed is obtained, and the matching error between the fourth coordinate data and the fifth coordinate data is determined; when the matching error is less than a preset error threshold, the spark region association mark between the first spark region to be confirmed and the second spark region to be confirmed is set as associative; when the first spark region to be confirmed and the second spark region to be confirmed are determined to be associative according to the spark region association mark, and the association cost when the first spark region to be confirmed and the second spark region to be confirmed are associated is less than a preset threshold, trajectory data is generated based on the first spark region to be confirmed and the second spark region to be confirmed. The structure score acquisition module is used to acquire a first structure nearest neighbor score corresponding to the trajectory data when the trajectory data is in a confirmed state; the first structure nearest neighbor score is used to characterize the proximity between the trajectory data and the structure reference point; The alarm information generation module is used to obtain the interference counter-evidence score value corresponding to the spark to be confirmed, and generate alarm prompt information when the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score and the interference counter-evidence score value. The interference counter-evidence score value is used to characterize the score value of the interference item that interferes with the spark to be confirmed as the target spark.

6. The apparatus according to claim 5, characterized in that, The device includes a loop module, used to, when there are other unprocessed multimedia data frames, set the next multimedia data frame to be processed as the current multimedia data frame to be processed, set the second unconfirmed spark region as the first unconfirmed spark region, obtain the next multimedia data frame to be processed from the current multimedia data frame to be processed; and continue to execute the step of obtaining the second unconfirmed spark region in the next multimedia data frame to be processed, until all multimedia data frames to be processed are processed.

7. The apparatus according to claim 5, characterized in that, The alarm information generation module is further configured to, based on the trajectory data, obtain the brightness feature data of the trajectory data in each of the multimedia data frames to be processed, and generate a brightness sequence according to the brightness feature data; perform frequency domain analysis on the brightness sequence to determine the first energy data corresponding to the preset target frequency band and the second energy data corresponding to the brightness sequence, and determine the energy ratio according to the first energy and the second energy. Obtain a preset brightness alternation template, and determine the correlation coefficient between the brightness sequence and the preset brightness alternation template based on the brightness sequence and the preset brightness alternation template; If the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the energy ratio, an alarm message is generated; or, if the spark to be confirmed is determined to be the target spark based on the first structural nearest neighbor score, the interference counter-evidence score, and the correlation coefficient, an alarm message is generated.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images