Video analysis method and device for power transmission line end, electronic equipment and storage medium

Abstract

The application provides a power transmission line end video analysis method and device, electronic equipment and storage medium, the method comprises the following steps: obtaining original video data obtained by shooting a power transmission line area based on a dual-spectrum thermal imager, preprocessing to obtain target video data; extracting target object information; determining the motion trajectory of the target object based on the target object information; cropping the target object to obtain a region of interest; inputting the motion trajectory of the target object and the region of interest into a video analysis model to obtain target object confidence, target object motion trajectory confidence and target object overlap degree; combining the target object confidence threshold, the motion trajectory confidence threshold and the target object overlap degree threshold to determine the target object detection result in the original video data. The application can solve the technical problems of difficult inspection, low inspection efficiency and waste of a large amount of manpower and material resources caused by manual inspection of the power transmission line in the prior art.

Description

Technical Field

[0001] This invention relates to the field of video analytics technology, specifically to a video analytics method, apparatus, electronic device, and storage medium for power transmission line terminals. Background Technology

[0002] Due to economic development, construction and renovation projects are frequent. Furthermore, power lines located in remote, isolated areas are highly susceptible to external damage, leading to a year-on-year increase in line tripping accidents. Traditional inspection methods can no longer meet current safety requirements. Therefore, the power industry urgently needs a powerful monitoring and surveillance system to monitor the surrounding conditions and environmental parameters of transmission lines around the clock, ensuring that transmission lines operate under visible and controllable conditions. Currently, monitoring is generally conducted manually, but some areas are difficult for inspectors to reach, resulting in difficulties in inspecting transmission lines, low inspection efficiency, and a waste of significant manpower and resources. Summary of the Invention

[0003] In view of this, it is necessary to provide a video analysis method, device, electronic equipment and storage medium for transmission line ends to solve the technical problems of difficult inspection, low inspection efficiency and waste of a lot of manpower and material resources caused by manual inspection of transmission lines in the prior art.

[0004] To achieve the above objectives, the present invention provides a video analysis method for transmission line ends, comprising: The raw video data of the transmission line area captured by the dual-spectrum thermal imager is acquired, and the raw video data is preprocessed to obtain the target video data. Extract the target object information from each video frame of the target video data; Based on the target object information corresponding to adjacent video frames in the target video data, the motion trajectory of the target object is determined; Based on the target object information of each video frame, the target object of each video frame is cropped to obtain the region of interest corresponding to each video frame. The motion trajectory of the target object and the region of interest corresponding to each video frame are input into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. Based on the target object confidence score, the target object trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, trajectory confidence thresholds, and target object overlap thresholds, the target object detection result in the original video data is determined.

[0005] Further, the preprocessing of the original video data to obtain the target video data includes: The original video data is decoded to obtain the original video frames; The original video frames are scaled according to the template size to obtain the first intermediate video frame; The first intermediate video frame is color space converted to obtain the second intermediate video frame; The second intermediate video frame is dropped or padded to obtain the third intermediate video frame. Noise is removed from the third intermediate video frame to obtain the target video data.

[0006] Further, the step of extracting target object information from each video frame of the target video data includes: The target video data is input into a preset target detection algorithm model to obtain target object information for each video frame in the target video data; the target object information includes the position and size of the target object.

[0007] Further, determining the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data includes: The target object information corresponding to multiple adjacent video frames in the target video data is processed by correlation or feature matching algorithms to obtain the corresponding matching results. Based on the matching results, the motion trajectory of the target object is determined.

[0008] Furthermore, the video analysis methods at the transmission line end also include: If the target detection result is determined to meet the preset alarm triggering conditions, an alarm action is executed based on the preset alarm format.

[0009] Further, determining the target object detection result in the original video data based on the target object confidence score, the target object trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, trajectory confidence thresholds, and target object overlap thresholds, includes: The target object confidence of each video frame is filtered based on a preset target object confidence threshold, and the target object overlap of each video frame is filtered based on a preset target object overlap threshold to determine the single-entity detection result of the target object. Based on a preset confidence threshold for motion trajectory, the confidence of the motion trajectory of the target object is filtered to obtain motion trajectories that meet the conditions. Based on the motion trajectory that meets the conditions, determine the action category of the target object and the corresponding action category confidence and action time window; Based on the target object's action category, the action category confidence level, the action time window, and preset action category confidence level thresholds and action time window thresholds, the action detection result of the target object is determined; Based on the individual object detection results and the motion detection results of the target object, the target object detection results in the original video data are obtained.

[0010] Furthermore, the classification model is one of Kalman filtering, correlation filtering, or multi-target tracking.

[0011] The present invention also provides a video analysis device for a transmission line end, comprising: The preprocessing module is used to acquire raw video data of the transmission line area captured by a dual-spectrum thermal imager, and to preprocess the raw video data to obtain target video data. The extraction module is used to extract target object information from each video frame of the target video data; The first determining module is used to determine the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data. The cropping module is used to crop the target object in each video frame based on the target object information of each video frame, so as to obtain the region of interest corresponding to each video frame. The analysis module is used to input the motion trajectory of the target object and the region of interest corresponding to each video frame into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. The second determining module is used to determine the target object detection result in the original video data based on the target object confidence, the target object motion trajectory confidence, and the target object overlap, as well as preset target object confidence thresholds, motion trajectory confidence thresholds, and target object overlap thresholds.

[0012] The present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the video analysis method for transmission line ends as described in any of the preceding claims.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the video analysis method for transmission line ends as described in any of the preceding claims.

[0014] The beneficial effects of the above implementation method are as follows: The present invention provides video data of the target area of ​​the transmission line captured by a dual-spectrum thermal imager, which is the raw video data. The dual-spectrum thermal imager is based on imaging with both infrared and visible light, so it can image both day and night. After preprocessing the acquired raw video data, the motion trajectory and region of interest of the target object are extracted. The motion trajectory of the target object and the region of interest corresponding to each video frame are input into a preset video analysis model for intelligent analysis to obtain multiple confidence levels. Finally, combined with a preset threshold, the detection result of the target object is determined. The entire process from data acquisition to data analysis is automated, which solves the technical problems of difficult inspection, low inspection efficiency, and waste of a lot of manpower and material resources caused by manual inspection of transmission lines in the prior art. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an embodiment of the video analysis method for power transmission line terminals provided by the present invention; Figure 2 A schematic diagram of a structure of an embodiment of the video analysis device at the end of a power transmission line provided by the present invention; Figure 3 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0019] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0020] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a video analysis method, apparatus, electronic device, and storage medium for transmission line terminals, which are described below.

[0023] like Figure 1 As shown, the present invention provides a video analysis method for a transmission line end, comprising: Step 110: Obtain the original video data of the transmission line area captured by the dual-spectrum thermal imager, and preprocess the original video data to obtain the target video data.

[0024] Step 120: Extract the target object information from each video frame of the target video data; Step 130: Determine the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data; Step 140: Based on the target object information of each video frame, crop the target object of each video frame to obtain the region of interest corresponding to each video frame. Step 150: Input the motion trajectory of the target object and the region of interest corresponding to each video frame into the preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. Step 160: Based on the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, motion trajectory confidence thresholds, and target object overlap score thresholds, determine the target object detection result in the original video data.

[0025] Understandably, the video data acquired by a dual-spectrum thermal imager of the target area of ​​the power transmission line is the raw video data. The dual-spectrum thermal imager is equipped with an uncooled thermal imaging detector with a resolution of 256 × 192, and its visible light channel and thermal imaging channel can be arbitrarily combined. The camera lens of the dual-spectrum thermal imager converts light signals into electrical signals, and the camera captures continuous image frames, forming a video input stream. Each image frame consists of pixels, containing values ​​from the RGB (red, green, and blue) color channels, forming a complete video stream input. Dual-spectrum refers to infrared and visible light; the dual-spectrum thermal imager images based on the combined use of infrared and visible light.

[0026] Visible light and thermal imaging can be visualized on the same platform, enabling AI (artificial intelligence) algorithm analysis and early warning of real-time video streams during the day, and combining AI algorithm analysis and early warning with thermal imaging algorithm analysis and alarm at night.

[0027] This invention can achieve real-time early warning of intelligent identification of vehicles such as excavators, bulldozers, dump trucks, pump trucks, cranes, cement mixers, tower cranes, and trucks around power transmission line corridors during the day and night, as well as early warning of wildfires within a certain range.

[0028] As an AI model, the video analytics model utilizes a novel model library based on massive amounts of data. Its engine employs 24 / 7 real-time video analysis, enabling more timely and accurate early warnings from front-end devices. Equipped with a 256*192 resolution vanadium oxide uncooled thermal imaging lens to identify wildfires within 2000 meters, it also utilizes visible light for AI analysis at night, truly achieving 24 / 7 protection of power transmission channels. Traditional point-and-shoot cameras are typically set to take photos every 30 or 60 minutes for intelligent image analysis, resulting in missed reports and delayed alarms. Unlike traditional image comparison solutions, the method provided in this invention uses AI analysis to prevent devices from missing important information during sleep periods.

[0029] The hardware utilizes an industrial-grade core processor framework, equipped with a high-performance CPU (Central Processing Unit) and an independent encoding system. The main camera features a 4MP starlight-level visible light lens, while the secondary camera employs a vanadium oxide uncooled thermal imaging lens. It incorporates a built-in full-network 4G industrial-grade module and supports 2G / 3G / 4G, intelligently switching network standards according to business needs. The system features a separate heartbeat and data interaction framework, an ultra-low power standby mode, and a combination of active and passive sleep / wake-up mechanisms. This invention supports remote video and image capture configurations, a 4MP starlight-level visible light lens, and multiple encoding formats such as H.265 / H.264. It transmits high-definition real-time video with ultra-low bandwidth, supports G.711a audio encoding, and enables two-way voice communication.

[0030] Specifically, the target object information of each video frame in the target video data is extracted, which is also known as target detection. Target detection involves using a target detection algorithm, such as a deep learning-based algorithm or a traditional feature extraction and classification algorithm, in the video frame to identify the target object in the image and locate its position and size.

[0031] The process of extracting the region of interest (ROI) is also known as object cropping, a technique that analyzes local regions of interest (ROIs) of a target object. It separates the target object from the background by cropping the image region of the target object in each video frame, and then feeds the cropped target image into an analysis model for further processing. The main steps include: Object detection: Perform object detection on video frames to identify the position and size of target objects.

[0032] Target cropping: Based on the detected target location, crop the target image region from the original image and remove background noise and other interference.

[0033] The advantage of target cropping is that it reduces the size of the input image, improves the computational efficiency of the analysis model, and can highlight the features of the target object by removing background noise, which is beneficial for subsequent analysis tasks such as object recognition and face recognition.

[0034] In video analytics and detection, confidence score is used to represent the degree of credibility of an algorithm's results for object detection, classification, or other tasks. It is a probability value or score, typically between 0 and 100, and can be understood as the algorithm's confidence or certainty regarding a particular result.

[0035] The role of confidence in video analysis is as follows: In object detection or classification tasks, confidence level represents how confident an algorithm is about the detected target or the classification result. For example, in object detection, for each detected target object, the algorithm provides a confidence level, representing the probability that the target is a real target. A high confidence level means that the algorithm is quite certain of the target's existence, while a low confidence level may indicate that the object detection or classification result is unreliable.

[0036] In target tracking tasks, confidence is used to represent the accuracy of the algorithm in tracking the target trajectory. Each frame's tracking box or trajectory is typically accompanied by a confidence score, reflecting the algorithm's confidence in the tracking. A higher confidence score indicates reliable tracking results, while a lower confidence score may indicate erroneous or uncertain target tracking.

[0037] By setting a threshold, results with high confidence can be filtered out, thereby improving the accuracy and reliability of the algorithm. For example, a confidence threshold can be set to retain only detection results with a confidence level higher than that threshold and discard results with lower confidence levels.

[0038] Setting pre-defined thresholds for comparison: In video analytics, setting pre-defined thresholds for comparison refers to using pre-defined thresholds or limits to filter and evaluate the algorithm's output. These pre-defined thresholds can be set based on the specific needs of the problem and the application scenario to control and optimize the results.

[0039] Target object confidence threshold: In target detection, a target object confidence threshold can be set to retain only detection results with a confidence level higher than this threshold. Results with a confidence level lower than this threshold are considered unreliable, thus filtering out high-quality detection results.

[0040] Overlap Threshold: In multi-object detection, an overlap threshold can be set to determine the degree of overlap between two object detection boxes (target boxes). If the overlap between two target boxes exceeds the set threshold, they may be considered duplicate detections of the same target, requiring merging or deduplication.

[0041] Motion trajectory confidence threshold (target tracking confidence threshold): During target object tracking, a confidence threshold can be set for each tracking result (the motion trajectory of the target object). If the confidence of the tracking result is lower than this threshold, the tracking can be considered unreliable, and restarting or deleting the tracking should be considered.

[0042] Long-term target tracking termination threshold: For tasks involving long-term tracking, a termination threshold can be set to determine whether the target object has disappeared or tracking has failed. If the target object fails to be tracked continuously for more than the set threshold within a certain period of time, the tracking can be terminated.

[0043] In some embodiments, the preprocessing of the original video data to obtain target video data includes: The original video data is decoded to obtain the original video frames; The original video frames are scaled according to the template size to obtain the first intermediate video frame; The first intermediate video frame is color space converted to obtain the second intermediate video frame; The second intermediate video frame is dropped or padded to obtain the third intermediate video frame. Noise is removed from the third intermediate video frame to obtain the target video data.

[0044] Understandably, video preprocessing aims to improve the effectiveness of subsequent analysis, and the processing steps include: Video decoding: Decodes the video stream, restoring the compressed encoding format (such as H.264) back to the original image frames.

[0045] Image scaling: Adjusting image frames to a specific size to fit the input requirements of subsequent analysis models. Typically, images are scaled to a fixed width and height, such as 224x224 pixels.

[0046] Color space conversion: Converting image frames from the RGB (red, green, blue) color space to other color spaces, such as grayscale images or the HSV (hue, saturation, brightness) color space. Images in different color spaces have different characteristics, which is helpful for subsequent analysis tasks.

[0047] Frame rate control: In real-time video analysis, due to limitations in computing resources or algorithm requirements, it may be necessary to drop or supplement video frames in order to control the frame rate of analysis and processing.

[0048] Noise Reduction: Denoising the image frames to remove interference noise and improve the accuracy of subsequent analysis.

[0049] In some embodiments, extracting target object information from each video frame of the target video data includes: The target video data is input into a preset target detection algorithm model to obtain target object information for each video frame in the target video data; the target object information includes the position and size of the target object.

[0050] In some embodiments, determining the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data includes: The target object information corresponding to multiple adjacent video frames in the target video data is processed by correlation or feature matching algorithms to obtain the corresponding matching results. Based on the matching results, the motion trajectory of the target object is determined.

[0051] Understandably, during target tracking, the target's motion trajectory is tracked across consecutive video frames by comparing the target detection results with those of previous frames and utilizing correlation or feature matching methods. Target tracking algorithms include those based on Kalman filtering, correlation filters, and multi-target tracking.

[0052] The advantage of object detection and tracking lies in its ability to handle targets in dynamic scenes and output their position trajectory. It is suitable for tasks that require real-time analysis of the dynamic changes of targets in videos, such as behavior analysis and target tracking.

[0053] In some embodiments, the video analysis method at the transmission line end further includes: If the target detection result is determined to meet the preset alarm triggering conditions, an alarm action is executed based on the preset alarm format.

[0054] Understandably, alarms typically send alert information to users or systems in the form of audio, video, text, or other means to notify users of a specific event or the detection of an anomaly. Specifically: 1. Alarm type: Alarm outputs can be selected according to the specific application scenario and task requirements. Common alarm types include: Sound alarm: Attracts the user's attention by playing preset sounds or alarm tones.

[0055] Video alarm: Mark or highlight abnormal areas or target objects in the video to attract the user's attention.

[0056] Text Alarm: Outputs alarm information in text format, such as via pop-up windows, emails, or SMS messages, to relevant personnel.

[0057] Graphical alarms: By drawing relevant graphics or markings, abnormal or important information can be displayed to users in a visual way.

[0058] 2. Alarm triggering conditions: The triggering conditions for alarm outputs are set according to specific tasks and application scenarios. They can be triggered based on preset rules, thresholds, or the output of algorithm models, for example: Target quantity threshold: An alarm is triggered when the number of detected targets exceeds the set threshold.

[0059] Target attribute judgment: An alarm is triggered when the target's attributes (such as shape and size) do not match the preset conditions.

[0060] Action recognition: Triggers an alarm when a specific action or behavior is recognized.

[0061] Anomaly detection: An alarm is triggered when an anomaly (such as external damage, smoke, flames, etc.) is detected.

[0062] 3. Alarm handling: Alarm outputs typically require corresponding alarm handling mechanisms. Generally, alarm handling may include the following steps: Alarm notification: Send alarm information to relevant personnel or departments, as well as other related systems.

[0063] Emergency Response: Take immediate action to respond to the alarm, such as contacting security personnel or dispatching police vehicles.

[0064] Log the alarm events: Record and archive alarm events for later review and analysis.

[0065] Alarm outputs play a crucial role in real-time video analytics, promptly alerting users to events or anomalies, thus supporting effective security monitoring and management. Alarm output settings must be adjusted according to specific needs and the environment to ensure timely and accurate alerts.

[0066] In some embodiments, determining the target object detection result in the original video data based on the target object confidence score, the target object trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, trajectory confidence thresholds, and target object overlap thresholds, includes: The target object confidence of each video frame is filtered based on a preset target object confidence threshold, and the target object overlap of each video frame is filtered based on a preset target object overlap threshold to determine the single-entity detection result of the target object. Based on a preset confidence threshold for motion trajectory, the confidence of the motion trajectory of the target object is filtered to obtain motion trajectories that meet the conditions. Based on the motion trajectory that meets the conditions, determine the action category of the target object and the corresponding action category confidence and action time window; Based on the target object's action category, the action category confidence level, the action time window, and preset action category confidence level thresholds and action time window thresholds, the action detection result of the target object is determined; Based on the individual object detection results and the motion detection results of the target object, the target object detection results in the original video data are obtained.

[0067] Understandably, the action category confidence threshold (action recognition threshold) is a threshold that can be set in behavior recognition to determine whether to accept or reject the recognition result of a certain action. Only when the confidence of an action is higher than this threshold is the action considered to be correctly recognized.

[0068] Action time window threshold: In behavior analysis, a time window threshold can be set to determine whether a certain behavior occurs within a certain duration. If a certain behavior occurs more times than the threshold within the set time window, the behavior can be considered successfully detected.

[0069] In summary, the video analysis method for transmission line ends provided by the present invention includes: acquiring raw video data of a transmission line area captured by a dual-spectrum thermal imager; preprocessing the raw video data to obtain target video data; extracting target object information from each video frame in the target video data; determining the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data; cropping the target object in each video frame based on the target object information in each video frame to obtain a region of interest corresponding to each video frame; inputting the motion trajectory of the target object and the region of interest corresponding to each video frame into a preset video analysis model to obtain the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score; and determining the target object detection result in the raw video data based on the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, motion trajectory confidence thresholds, and target object overlap thresholds.

[0070] This invention acquires video data of the target area of ​​a power transmission line using a dual-spectrum thermal imager, which serves as the raw video data. The dual-spectrum thermal imager uses both infrared and visible light for imaging, allowing for imaging both day and night. After preprocessing the acquired raw video data, the motion trajectory and region of interest (ROI) of the target object are extracted. The motion trajectory and ROI corresponding to each video frame are then input into a preset video analysis model for intelligent analysis, yielding multiple confidence levels. Finally, combined with preset thresholds, the detection result of the target object is determined. The entire process, from data acquisition to data analysis, is automated, solving the technical problems of difficult, inefficient, and wasteful manual inspections of power transmission lines in existing technologies.

[0071] like Figure 2 As shown, the present invention also provides a video analysis device 200 at the end of a transmission line, which includes: Preprocessing module 210 is used to acquire raw video data of the transmission line area captured by a dual-spectrum thermal imager, and preprocess the raw video data to obtain target video data. Extraction module 220 is used to extract target object information from each video frame in the target video data; The first determining module 230 is used to determine the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data. The cropping module 240 is used to crop the target object of each video frame based on the target object information of each video frame to obtain the region of interest corresponding to each video frame. Analysis module 250 is used to input the motion trajectory of the target object and the region of interest corresponding to each video frame into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. The second determining module 260 is used to determine the target object detection result in the original video data based on the target object confidence, the target object motion trajectory confidence, and the target object overlap, as well as preset target object confidence thresholds, motion trajectory confidence thresholds, and target object overlap thresholds.

[0072] The video analysis device at the transmission line end provided in the above embodiments can realize the technical solutions described in the above embodiments of the video analysis method at the transmission line end. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the video analysis method at the transmission line end, and will not be repeated here.

[0073] like Figure 3 As shown, the present invention also provides an electronic device 300. The electronic device 300 includes a processor 301, a memory 302, and a display 303. Figure 3 Only some components of the electronic device 300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0074] In some embodiments, memory 302 may be an internal storage unit of electronic device 300, such as a hard disk or memory of electronic device 300. In other embodiments, memory 302 may also be an external storage device of electronic device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 300.

[0075] Furthermore, the memory 302 may include both internal storage units of the electronic device 300 and external storage devices. The memory 302 is used to store application software and various types of data installed on the electronic device 300.

[0076] In some embodiments, processor 301 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 302 or process data, such as the video analysis method at the transmission line end in this invention.

[0077] In some embodiments, display 303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 303 is used to display information from electronic device 300 and to display a visual user interface. Components 301-303 of electronic device 300 communicate with each other via a system bus.

[0078] In some embodiments of the present invention, when the processor 301 executes the video analysis program for the transmission line end in the memory 302, the following steps can be implemented: The raw video data of the transmission line area captured by the dual-spectrum thermal imager is acquired, and the raw video data is preprocessed to obtain the target video data. Extract the target object information from each video frame of the target video data; Based on the target object information corresponding to adjacent video frames in the target video data, the motion trajectory of the target object is determined; Based on the target object information of each video frame, the target object of each video frame is cropped to obtain the region of interest corresponding to each video frame. The motion trajectory of the target object and the region of interest corresponding to each video frame are input into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. Based on the target object confidence score, the target object trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, trajectory confidence thresholds, and target object overlap thresholds, the target object detection result in the original video data is determined.

[0079] It should be understood that when the processor 301 executes the video analysis program at the transmission line end in the memory 302, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0080] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 300 mentioned. Electronic device 300 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, electronic device 300 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0081] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the video analysis method for the transmission line end provided by the methods described above, the method comprising: The raw video data of the transmission line area captured by the dual-spectrum thermal imager is acquired, and the raw video data is preprocessed to obtain the target video data. Extract the target object information from each video frame of the target video data; Based on the target object information corresponding to adjacent video frames in the target video data, the motion trajectory of the target object is determined; Based on the target object information of each video frame, the target object of each video frame is cropped to obtain the region of interest corresponding to each video frame. The motion trajectory of the target object and the region of interest corresponding to each video frame are input into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. Based on the target object confidence score, the target object trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, trajectory confidence thresholds, and target object overlap thresholds, the target object detection result in the original video data is determined.

[0082] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0083] The video analysis method, apparatus, electronic device, and storage medium for transmission line terminals provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A video analysis method for a transmission line end, characterized in that, include: The raw video data of the transmission line area captured by the dual-spectrum thermal imager is acquired, and the raw video data is preprocessed to obtain the target video data. Extracting target object information from each video frame of the target video data includes: inputting the target video data into a preset target detection algorithm model to obtain target object information for each video frame of the target video data; the target object information includes the position and size of the target object; Based on the target object information corresponding to adjacent video frames in the target video data, the motion trajectory of the target object is determined; Based on the target object information of each video frame, the target object of each video frame is cropped to obtain the region of interest corresponding to each video frame. The motion trajectory of the target object and the region of interest corresponding to each video frame are input into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. Based on the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, motion trajectory confidence thresholds, and target object overlap score thresholds, the target object detection result in the original video data is determined. The determination of the target object detection result in the original video data based on the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, motion trajectory confidence thresholds, and target object overlap score thresholds, includes: The target object confidence of each video frame is filtered based on a preset target object confidence threshold, and the target object overlap of each video frame is filtered based on a preset target object overlap threshold to determine the single-entity detection result of the target object. Based on a preset confidence threshold for motion trajectory, the confidence of the motion trajectory of the target object is filtered to obtain motion trajectories that meet the conditions. Based on the motion trajectory that meets the conditions, determine the action category of the target object and the corresponding action category confidence and action time window; Based on the target object's action category, the action category confidence level, the action time window, and preset action category confidence level thresholds and action time window thresholds, the action detection result of the target object is determined; Based on the individual object detection results and the motion detection results of the target object, the target object detection results in the original video data are obtained.

2. The video analysis method for transmission line ends according to claim 1, characterized in that, The preprocessing of the original video data to obtain the target video data includes: The original video data is decoded to obtain the original video frames; The original video frames are scaled according to the template size to obtain the first intermediate video frame; The first intermediate video frame is color space converted to obtain the second intermediate video frame; The second intermediate video frame is dropped or padded to obtain the third intermediate video frame. Noise is removed from the third intermediate video frame to obtain the target video data.

3. The video analysis method for transmission line ends according to claim 1, characterized in that, Determining the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data includes: The target object information corresponding to multiple adjacent video frames in the target video data is processed by correlation or feature matching algorithms to obtain the corresponding matching results. Based on the matching results, the motion trajectory of the target object is determined.

4. The video analysis method for transmission line ends according to claim 1, characterized in that, Also includes: If the target detection result is determined to meet the preset alarm triggering conditions, an alarm action is executed based on the preset alarm format.

5. The video analysis method for transmission line ends according to any one of claims 1-4, characterized in that, The video analysis model is one of Kalman filtering, correlation filtering, or multi-target tracking.

6. A video analysis device for a transmission line end, characterized in that, include: The preprocessing module is used to acquire raw video data of the transmission line area captured by a dual-spectrum thermal imager, and to preprocess the raw video data to obtain target video data. An extraction module is used to extract target object information from each video frame of the target video data, including: inputting the target video data into a preset target detection algorithm model to obtain target object information for each video frame of the target video data; the target object information includes the position and size of the target object; The first determining module is used to determine the motion trajectory of the target object based on the target object information corresponding to adjacent video frames in the target video data. The cropping module is used to crop the target object in each video frame based on the target object information of each video frame, so as to obtain the region of interest corresponding to each video frame. The analysis module is used to input the motion trajectory of the target object and the region of interest corresponding to each video frame into a preset video analysis model to obtain the target object confidence, the target object motion trajectory confidence, and the target object overlap. The second determining module is used to determine the target object detection result in the original video data based on the target object confidence score, the target object motion trajectory confidence score, and the target object overlap score, as well as preset target object confidence score thresholds, motion trajectory confidence thresholds, and target object overlap thresholds, including: The target object confidence of each video frame is filtered based on a preset target object confidence threshold, and the target object overlap of each video frame is filtered based on a preset target object overlap threshold to determine the single-entity detection result of the target object. Based on a preset confidence threshold for motion trajectory, the confidence of the motion trajectory of the target object is filtered to obtain motion trajectories that meet the conditions. Based on the motion trajectory that meets the conditions, determine the action category of the target object and the corresponding action category confidence and action time window; Based on the target object's action category, the action category confidence level, the action time window, and preset action category confidence level thresholds and action time window thresholds, the action detection result of the target object is determined; Based on the individual object detection results and the motion detection results of the target object, the target object detection results in the original video data are obtained.

7. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the video analysis method for a transmission line end as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores a computer program that, when executed by a processor, implements the video analysis method for transmission line ends as described in any one of claims 1 to 5.

Images