Foreign Object Detection Method for Substation Safety Monitoring Videos

Abstract

The present invention relates to the technical field of substation monitoring, and specifically relates to a foreign object detection method for substation safety monitoring videos. When this method performs foreign object recognition, it will be compared with the static background frame bk and the dynamic background frame bk1 respectively, and then the foreground object points will be determined for subsequent detection; moreover, the dynamic background frame bk1 will be updated in real time during the detection process. Specifically, if there are no foreground object points in a detected image g, then this image g will be updated as the latest dynamic background frame bk1. In other words, the dynamic background frame bk1 will change in real time according to the dynamic changes such as light in the environment. In this way, through the combined action of the initial static background frame bk and the dynamic background frame bk1, while ensuring accurate recognition of the target area, it can also avoid the influence of slow changes in light in the environment on the detection. The key point of this method lies in the idea, and the specific comparison and recognition of images can adopt conventional image processing methods, with strong portability and very good applicability.

Description

Technical Field

[0001] The present invention relates to the technical field of substation monitoring, and particularly to a foreign object detection method for substation security monitoring videos. Background Art

[0002] The working environment of a substation has relatively high requirements for safety. In order to ensure that there is sufficient insulation distance between equipment in a high-voltage working environment and that there are no dangerous situations such as short circuits and overvoltages in the lines and equipment, any foreign objects (leftovers) should be avoided around its working equipment, and even more so, there should be no malicious sabotage by humans.

[0003] In recent years, with the improvement of monitoring video and real-time target detection technologies, the monitoring of substations has also become intelligent, and the monitoring of foreign objects has thus become more convenient. In current substations, unidirectional transmission of video signals can be achieved using a video monitoring system in a bandwidth environment below 2M. In addition, the video monitoring system in a 500kV substation also supports the motion detection function, and the camera images of any area can be retrieved through the monitor of the monitoring master station. Since the video monitoring system was applied in the substation, the monitoring efficiency has been increasingly improved, and the monitoring effect has become more ideal. Currently, the personnel working in the substation control room can use the video monitoring system to achieve the supervision and control of the entire power system, and power enterprises developing the monitoring system can solve the problem of insufficient personnel and save personnel costs.

[0004] In a security monitoring system with a camera as the video acquisition device, foreign object detection usually adopts the method of target detection. Target detection refers to detecting the changed area in a sequence of images and detecting the target from the background image. Usually, after target classification, tracking, and behavior understanding, etc., the post-processing process only considers the pixel area corresponding to the target in the image. Therefore, the correct detection and segmentation of the target are very important for the subsequent processing. During specific detection, according to whether the camera remains stationary, the detection can be divided into two categories: static background and moving background. Most video monitoring systems have fixed cameras. Therefore, the moving target detection method under a static background has received extensive attention. Currently, the commonly used moving target detection methods under a static background are the frame difference method, the optical flow method, and the background subtraction method. However, in the actual application of substation monitoring, due to the dynamic changes in the scene, such as the influence of weather, light, shadows, and cluttered background interference, etc., it makes target detection and segmentation quite difficult. In other words, most existing automatic detection systems are difficult to adapt to situations such as light changes and occlusions, which leads to the low accuracy of existing detection methods under the interference of a dynamic environment.

[0005] Therefore, under the interference of a dynamic environment, how to ensure the accuracy of foreign object detection has become an urgent problem to be solved currently. Summary of the Invention

[0006] In view of the deficiencies of the above-mentioned existing technologies, the present invention provides a foreign object detection method for substation security monitoring videos, which can still ensure the accuracy of foreign object detection under the interference of dynamic environments.

[0007] To solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A foreign object detection method for substation security monitoring videos includes the following steps:

[0009] S1. Obtain three process reference images fo, fo1, and fg with the same image size as the historical video stream, and set the initial attribute values of each point in fo, fo1, and fg to 0; then screen out two frames of images from the images of the historical video stream and use them as the static background frame bk and the dynamic background frame bk1 respectively.

[0010] S2. After extracting a frame of image g to be detected from the video stream to be measured according to the preset frame extraction method; compare each point in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively. If there is a foreground object point in g, record the attribute value of the corresponding point in fo as an abnormal value; if there is no point with an abnormal attribute value in fo, go to S3, otherwise go to S4.

[0011] S3. Use the image g as the updated dynamic background frame bk1 and return to S2.

[0012] S4. Screen out the points with abnormal attribute values in fo according to the preset abnormal point screening method, and use the points after screening as suspected points.

[0013] S5. Add 1 to the attribute value of the point in fg corresponding to each suspected point, and record the number of times each point is marked as a suspected point through the attribute value of each point in fg.

[0014] S6. Statistically analyze the growth value A of the attribute value of each point in fg within the preset time period, and judge whether each point is a foreign object point according to the preset foreign object judgment method based on the growth value A of each point. If so, record the attribute value of the corresponding point in fo1 as a foreign object value.

[0015] S7. Analyze whether there is a point with a foreign object value in fo1. If not, return to S2; if so, after reporting the foreign object, return to S2.

[0016] Principle and effect of the basic solution:

[0017] 1. When the foreign object recognition is performed by this method, it will be compared with the static background frame bk and the dynamic background frame bk1 respectively, and then the foreground object points (i.e., abnormal points) will be determined for subsequent detection; moreover, the dynamic background frame bk1 will be updated in real time during the detection process. Specifically, if there are no foreground object points in a detected image g, then this image g will be updated as the latest dynamic background frame bk1. In other words, the dynamic background frame bk1 will change in real time according to the dynamic changes such as light in the environment. In this way, through the combined action of the initial static background frame bk and the dynamic background frame bk1, while ensuring the accurate recognition of the target area, it can also avoid the influence of the slow change of light in the environment on the detection.

[0018] 2. After being compared with the static background frame bk and the dynamic background frame bk1 respectively and identifying the foreground object, abnormal point processing will be carried out to determine the specific suspected points, which can further ensure the accuracy of foreign object recognition.

[0019] 3. The key point of this method lies in the ingenuity of the idea, and the specific comparison and recognition of images can adopt conventional image processing methods, which has strong portability, low requirements for the performance of the processor, and very good applicability.

[0020] In summary, this method can still ensure the accuracy rate of foreign object detection under the interference of a dynamic environment.

[0021] Preferably, in S2, when comparing the points in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively, first calculate the difference between the pixel of the point in the image g and the pixel of the corresponding point in the static background frame bk; if the absolute value of the difference is greater than or equal to the set value, then calculate the difference between the pixel of this point and the pixel of the corresponding point in the dynamic background frame bk1. If the absolute value of the difference is still greater than the set value, then this point is recorded as a foreground object point.

[0022] Beneficial effects: The way of separate comparison can make full use of the specific features of the static background frame bk and the dynamic background frame bk1. While ensuring the recognition efficiency, it can also ensure the recognition accuracy of the foreground object points.

[0023] Preferably, S4 includes:

[0024] S41. Perform an opening operation on fo to obtain fo01;

[0025] S42. Obtain the connected regions of the points with abnormal attribute values in fo01, and perform a preset connected region inspection and screening. Denote the remaining connected regions after the inspection and screening as effective connected regions;

[0026] S43. Perform a preset texture feature verification on the regions in the image g corresponding to each effective connected region, and denote the points in the effective connected regions corresponding to the remaining regions after passing the texture feature verification as suspected points.

[0027] Beneficial effects: Through opening operation, the misdetection areas caused by interference can be reduced; by means of overall screening of connected regions, some interference items that are obviously not foreign objects can be removed according to the actual situation, thus ensuring the accuracy of recognition.

[0028] Preferably, in S41, after performing opening operation and closing operation on fo, fo01 is obtained.

[0029] Beneficial effects: After adding the closing operation, it can also reduce the situation where a single target is broken into multiple parts due to being misdetected as the background because the pixels in the same target are close to the background pixels.

[0030] Preferably, in S42, the preset connected region inspection and screening include: for each connected region in fo01, record the coordinates of all points with abnormal attribute values in the connected region, and calculate the number of points with abnormal attribute values in the connected region. If the number is less than the preset low threshold or greater than the preset high threshold, then screen out the connected region.

[0031] Preferably, in S42, the preset connected region inspection and screening also include: record the vertex coordinates of the circumscribed rectangle of each connected region, and after calculating the aspect ratio of the circumscribed rectangle according to the vertex coordinates of the circumscribed rectangle of the connected region, screen out the connected regions with an aspect ratio greater than the preset ratio.

[0032] Preferably, in S42, the preset connected region inspection and screening also include: after calculating the area of the circumscribed rectangle according to the vertex coordinates of the circumscribed rectangle of the connected region, calculate the ratio of the area occupied by the points with abnormal attribute values in the connected region to the area of the circumscribed rectangle of the connected region, and perform shape constraint verification on the detected target according to this ratio and the aspect ratio of the circumscribed rectangle of the connected region. If the result of the constraint verification does not belong to a foreign object, then screen out the corresponding connected region.

[0033] Beneficial effects: Through the preset connected region inspection and screening, the connected regions that do not meet the foreign object conditions can be screened out, thus ensuring the accuracy of the finally detected foreign objects, and there is no need for manual screening of the finally detected foreign objects, which can not only reduce the workload of the staff, but also effectively avoid the situation of mistakes in foreign object reporting (such as noise points or misidentification of normal devices).

[0034] Preferably, in S43, the preset texture feature verification includes:

[0035] S431. Respectively extract the texture maps of the regions corresponding to each effective connected region in the image g, the static background frame bk and the dynamic background frame bk1. When extracting the texture map, the texture of a point is the average value of the absolute value of the difference between this point and its left point and the absolute value of the difference between this point and its upper point;

[0036] S432. Extract features from each texture map. Draw a horizontal dividing line and a vertical dividing line from the center of the texture map to divide the area into four equal-sized sub-regions, and label the area numbers of the four sub-regions as 1, 2, 3, and 4 respectively. After summing the texture map values in each sub-region, sort them in descending order. Then, obtain the feature value of the texture map according to a preset feature value calculation method. The feature value calculation method is: the area number ranked first * 1000 + the area number ranked second * 100 + the area number ranked third * 10 + the area number ranked fourth * 1;

[0037] S433. Analyze the feature values of the texture maps. If the feature value of a texture map of image g is the same as the feature value of the corresponding texture map of the static background frame bk or the dynamic background frame bk1, then screen out the corresponding effective detection area in fo01.

[0038] Beneficial effects: By extracting textures and calculating feature values, if the feature value of the area corresponding to the texture of the effective connected area in image g is the same as any one of the feature values of the corresponding positions of the two background frames, it is considered as the background, and the false detection interference can be excluded again. For example, when the shadow of a flying object in the sky is recognized as an abnormal point set, in this way, these objects that actually do not cause interference to the substation can be detected and screened out. After this step, a more accurate target connected area map can be obtained.

[0039] Preferably, in S431, when extracting the texture maps of the areas corresponding to the effective connected areas in image g, the static background frame bk or the dynamic background frame bk1, only extract the texture map of the central position area at the exact center. The length of the central position area is 1 / n of the corresponding effective connected area, and the width of the central position area is 1 / n of the corresponding effective connected area.

[0040] Beneficial effects: In this operation method, since the central position area encompasses the characteristics of each sub-region, while ensuring the accuracy of the texture feature values, the amount of calculation can be reduced, thereby effectively improving the detection efficiency.

[0041] Preferably, in S6, the preset foreign object judgment method includes: judging whether each point in image g is a suspected point in fo01. If it is a suspected point, then judge whether the growth value A of this point plus 1 is greater than or equal to N. If so, regard this point as a foreign object point; if not, regard this point as a normal point. Wherein, N is a preset first threshold value;

[0042] If it is not a suspected point, determine whether the growth value A of this point is 0. If it is 0, then regard this point as a normal point. If it is not 0, then subtract 1 from the growth value A of this point and then determine whether it is greater than M. If it is greater than M and the detection result of this point in the previous frame is a suspected point, then regard this point as a foreign object point. If it is less than or equal to M or the detection result of this point in the previous frame is not a suspected point, then regard this point as a normal point; where M is a preset second threshold, and M < N.

[0043] Beneficial effects: In the existing monitoring logic, usually if a suspected point continuously exists for more than a preset time, then this point is regarded as a foreign object point. Therefore, the existing foreign object point judgment logic usually is that when the number of times a certain point is continuously recorded as a suspected point exceeds a preset value, it is regarded as a foreign object point. The advantages of this are simple to implement and have high accuracy. However, such a detection method has the following problems: When detecting foreign objects, if the foreign object is temporarily blocked (such as when a staff member passes by a certain area and just blocks the corresponding field of view of the main camera), then this foreign object will not be marked as a suspected point during the corresponding time period. When this situation occurs, the conventional detection logic is to wait until the staff member no longer blocks and then re - count, which will greatly affect the timeliness of foreign object detection. And the longer a foreign object exists, the more potential hidden dangers there are for the safety of the substation. In the present invention, if a certain point belongs to a suspected point in the current detection frame, then the A of this point will be incremented by 1, and then it is determined whether the foreign object condition is met. In this way, even if a staff member temporarily blocks the foreign object, there is no need to re - count, thus ensuring the timeliness of foreign object detection and preventing the situation of missing detection of an existing foreign object for a long time.

[0044] In addition, if a point is no longer a suspected point at the current moment and the corresponding attribute value of this point in fg01 is not 0, this method will subtract 1 from the A of this point and then determine whether it is still greater than M after subtraction. If it is greater than M and the detection result of this point in the previous frame is a suspected point, it means that the time this point is judged as a suspected point is relatively long, and it does not belong to the situation where a staff member temporarily places and then takes it away, but is only temporarily blocked by the staff member in the current detection frame. Therefore, this point is regarded as a foreign object point; if it is less than or equal to M, it means that the time this point is judged as a suspected point is relatively short and it is not yet possible to accurately judge whether it is a foreign object point. In addition, if the detection result of this point in the previous frame is not a suspected point (that is, both of the two continuously extracted frames are not suspected points), it means that it is only temporarily placed by the staff member and has now been taken away. Therefore, this point is regarded as a normal point. Such a processing method can avoid mis - detection of objects (such as maintenance tools) carried or temporarily placed by staff members in the monitoring area, and further ensure the accuracy of the inspection results. Description of the Drawings

[0045] In order to make the purpose, technical solutions and advantages of the invention clearer, the present invention will be further described in detail below with reference to the drawings, where:

[0046] Figure 1 is the flowchart of the embodiment;

[0047] Figure 2 is the initial background frame of the operation example in the embodiment;

[0048] Figure 3 is the detection frame of the operation example in the embodiment;

[0049] Figure 4 is the detection result of the operation example in the embodiment. Specific implementation manner

[0050] The following is a further detailed description through specific implementation manners:

[0051] Embodiment:

[0052] As Figure 1 shown, a foreign object detection method for substation security monitoring video is disclosed in this embodiment, including the following steps:

[0053] S1. Monitoring initialization settings; after analyzing the image resolution of the historical video stream, set three process reference diagrams fo, fo1, and fg with the same image size as the historical video stream, and set the initial attribute values of each point in fo, fo1, and fg to 0; then screen out two frames of images from the images of the historical video stream, and use them as the static background frame bk and the dynamic background frame bk1 respectively; when selecting images, to ensure the accuracy and effectiveness of recognition, two frames of images without any foreign objects can be selected by the staff. Of course, if there are objects on the selected images that do not belong to the objects with potential safety hazards, they can also be used as the background frames, but the accuracy during use will be slightly affected, and they can still be used normally.

[0054] S2. After extracting a frame of image g to be detected from the video stream to be measured according to the preset frame extraction method; compare each point in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively. If there are foreground object points in g, record the attribute value of the corresponding point in fo as an abnormal value; if there are no points with abnormal attribute values in fo, go to S3, otherwise go to S4. The specific frame extraction method (mainly referring to the frame extraction frequency) can be specifically set by those skilled in the art according to the specific scale of the substation and the daily maintenance and inspection situation, and will not be elaborated here.

[0055] In specific implementation, when comparing the points in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively, first calculate the difference between the pixel of the point in the image g and the pixel of the corresponding point in the static background frame bk; if the absolute value of the difference is greater than or equal to the set value, then calculate the difference between the pixel of this point and the pixel of the corresponding point in the dynamic background frame bk1. If the absolute value of the difference is still greater than the set value, then mark this point as a foreground object point. The method of separate comparison can make full use of the specific features of the static background frame bk and the dynamic background frame bk1, and while ensuring the recognition efficiency, it can also ensure the recognition accuracy of the foreground object points.

[0056] S3. Update the dynamic background frame bk1, take the image g as the updated dynamic background frame bk1, and return to S2;

[0057] S4. Screen out the points with abnormal attribute values in fo according to the preset abnormal point screening method, and take the points after screening as suspected points. Specifically, S4 includes:

[0058] S41. After performing opening operation and closing operation on fo, obtain fo01. Through the opening operation, the false detection area caused by interference can be reduced; after adding the closing operation, it can also reduce the situation that the same target is misdetected as the background due to the proximity of the background pixels, resulting in a single target being broken into multiple parts.

[0059] S42. Obtain the connected regions of the points with abnormal attribute values in fo01, and perform the preset connected region inspection and screening. Denote the remaining connected regions after inspection and screening as effective connected regions;

[0060] In specific implementation, the preset connected region inspection and screening includes: for each connected region in fo01, record the coordinates of all points with abnormal attribute values in this connected region, and calculate the number of points with abnormal attribute values in the connected region. If the number is less than the preset low threshold or greater than the preset high threshold, then screen out this connected region; record the vertex coordinates of the circumscribed rectangle of each connected region, and calculate the aspect ratio of the circumscribed rectangle according to the vertex coordinates of the circumscribed rectangle of the connected region, and then screen out the connected regions with an aspect ratio greater than the preset ratio; calculate the area of the circumscribed rectangle according to the vertex coordinates of the circumscribed rectangle of the connected region, then calculate the ratio of the area occupied by the points with abnormal attribute values in the connected region to the area of the circumscribed rectangle of the connected region, and perform shape constraint verification on the detection target according to this ratio and the aspect ratio of the circumscribed rectangle of this connected region. If the result of the constraint verification is not a foreign object, then screen out the corresponding connected region.

[0061] Through the preset connected region inspection and screening, the connected regions that do not meet the foreign object conditions can be screened out, so as to ensure the accuracy of the finally detected foreign objects. There is no need for manual screening of the finally detected foreign objects, which can not only reduce the workload of the staff, but also effectively avoid mistakes in foreign object reporting (such as noise points or misidentification of normal devices).

[0062] S43. Perform a preset texture feature check on the regions corresponding to each effective connected region in the image g, and mark the points in the effective connected region corresponding to the remaining regions after passing the texture feature check as suspected points.

[0063] Specifically, when implemented, the preset texture feature check includes:

[0064] S431. Respectively extract the texture maps of the regions corresponding to each effective connected region in the image g, the static background frame bk, and the dynamic background frame bk1. When extracting the texture map, the texture of a point is the average value of the absolute value of the difference between the point and its left point and the absolute value of the difference between the point and its upper point. Among them, when extracting the texture map of the region corresponding to each effective connected region in the image g, the static background frame bk, or the dynamic background frame bk1, only extract the texture map of the central position region at the center. The length of the central position region is 1 / n of the corresponding effective connected region, and the width of the central position region is 1 / n of the corresponding effective connected region. Since the central position region encompasses the characteristics of each partition, such an operation method can reduce the computational amount while ensuring the accuracy of the texture feature value, thereby effectively improving the detection efficiency.

[0065] S432. Perform feature extraction on each texture map. Draw a horizontal dividing line and a vertical dividing line from the center of the texture map to divide the region into four sub-regions of equal size, and record the area numbers of the four sub-regions as 1, 2, 3, and 4 respectively; after summing the texture map values in each sub-region, sort them in descending order; then, obtain the feature value of the texture map according to the preset feature value calculation method; the feature value calculation method is the area number ranked first * 1000 + the area number ranked second * 100 + the area number ranked third * 10 + the area number ranked fourth * 1.

[0066] S433. Analyze the feature value of the texture map. If the feature value of a texture map in the image g is the same as the feature value of the corresponding texture map in the static background frame bk or the dynamic background frame bk1, then screen out the corresponding effective detection region in fo01.

[0067] By extracting the texture and calculating the eigenvalue, if the eigenvalue of the texture in the effective connected region corresponding to the region in image g is the same as any one of the eigenvalues at the corresponding positions of two background frames, it is considered as the background, and the false detection interference can be excluded again. For example, when the shadow of a flying object in the sky is recognized as a set of abnormal points, in this way, these objects that actually do not cause interference to the substation can be identified and filtered out. After this step, a more accurate target connected region map can be obtained.

[0068] S5. Add 1 to the attribute value of the point corresponding to each suspected point in fg, and record the number of times each point is marked as a suspected point through the attribute value of each point in fg.

[0069] S6. Statistically calculate the growth value A of the attribute value of each point in fg within a preset time period, and determine whether each point is a foreign object point according to the A of each point by a preset foreign object judgment method. If so, record the attribute value of the corresponding point in fo1 as the foreign object value.

[0070] Specifically in implementation, in S6, the preset foreign object judgment method includes: judging whether each point in image g is a suspected point in fo01. If it is a suspected point, then judge whether the growth value A of this point plus 1 is greater than or equal to N. If so, take this point as a foreign object point; if not, take this point as a normal point. Among them, N is a preset first threshold.

[0071] If it is not a suspected point, then judge whether the growth value A of this point is 0. If it is 0, take this point as a normal point; if it is not 0, then subtract 1 from the growth value A of this point and judge whether it is greater than M. If it is greater than M and the detection result of this point in the previous frame is a suspected point, then take this point as a foreign object point; if it is less than or equal to M or the detection result of this point in the previous frame is not a suspected point, then take this point as a normal point. Among them, M is a preset second threshold, and M < N.

[0072] In the existing monitoring logic, usually when the continuous existence time of a certain suspected point exceeds the preset time, this point is regarded as a foreign object point. Therefore, in the existing foreign object point judgment logic, usually when the number of times a certain point is continuously recorded as a suspected point exceeds the preset value, it is regarded as a foreign object point. The advantages of this are simple and easy to implement and high accuracy. However, there are the following problems with such a detection method: When detecting foreign objects, if the foreign object is temporarily blocked (for example, when a staff member passes by a certain area and just blocks the corresponding field of view of the main camera), then the foreign object will not be marked as a suspected point during the corresponding time period. When this situation occurs, the conventional detection logic is to wait until the staff member no longer blocks and then re-count, which will greatly affect the timeliness of foreign object detection. And for each additional minute that a foreign object exists, there will be one more potential hidden danger to the safety of the substation. In the present invention, if a certain point belongs to a suspected point in the current detection frame, the value of A of this point will be incremented by 1, and then it is judged whether the conditions for a foreign object are met. In this way, even if a staff member temporarily blocks the foreign object, there is no need to re-count, thus ensuring the timeliness of foreign object detection and preventing the situation where a foreign object already exists but is undetected for a long time.

[0073] In addition, if a point is no longer a suspected point at the current moment and the corresponding attribute value of this point in fg01 is not 0, this method will decrement the A of this point by 1 and then judge whether it is still greater than M after the decrement. If it is greater than M and the detection result of this point in the previous frame is a suspected point, it means that the time for this point to be judged as a suspected point is relatively long and it does not belong to the situation where a staff member temporarily places and then takes it away, but is only temporarily blocked by the staff member in the current detection frame. Therefore, this point is regarded as a foreign object point; if it is less than or equal to M, it means that the time for this point to be judged as a suspected point is relatively short and it is not yet possible to accurately judge whether it is a foreign object point. In addition, if the detection result of this point in the previous frame is not a suspected point (that is, both of the two continuously extracted frames are not suspected points), it means that it was only temporarily placed by the staff member and has now been taken away. Therefore, this point is regarded as a normal point. Such a processing method can avoid misdetection of objects (such as maintenance tools) carried or temporarily placed by staff members in the monitoring area, and further ensure the accuracy of the inspection results.

[0074] S7. Analyze whether there is a point in fo1 whose attribute value is the foreign object value. If not, return to S2; if so, after reporting the foreign object, then return to S2.

[0075] When the present method performs foreign object recognition, it will compare with the static background frame bk and the dynamic background frame bk1 respectively, and then determine the foreground object points (i.e., abnormal points) and perform subsequent detections; moreover, the dynamic background frame bk1 will be updated in real time during the detection process. Specifically, if there are no foreground object points in a detected image g, then this image g will be updated as the latest dynamic background frame bk1. In other words, the dynamic background frame bk1 will change in real time according to the dynamic changes such as light in the environment. In this way, through the combined effect of the initial static background frame bk and the dynamic background frame bk1, while ensuring the accurate recognition of the target area, it can also avoid the influence of the slow change of light in the environment on the detection. Moreover, the key point of this method lies in the ingenious idea, and the specific comparison and recognition of images can adopt conventional image processing methods, which has strong portability and low requirements for the performance of the processor, and has very strong applicability.

[0076] For the convenience of understanding, a specific operation example is used for illustration. Two scenes in the substation are selected (the line part and the main transformer part, the line part is Example 1, and the main transformer part is Example 2), and the image size of both is 704*576. Figure 2 are the initial background frames of the two scenes. Initially, the static background frame bk and the dynamic background frame bk1 are the same, both being the initial background frame; Figure 3 are the detection frames of the two scenes. Set the width W = 704, the height H = 576, the number of allowed occlusion frames C = 20, and the number of frames for determining a foreign object N = 100, then M = N - C = 100 - 20 = 80.

[0077] The input R, G, B images are converted into Y, Cr, Cb using the following formula.

[0078] Y = 0.299R + 0.587G + 0.114B (1);

[0079] Cr = (0.500R - 0.4187G - 0.0813B) + 128 (2);

[0080] Cb = (-0.1687R - 0.3313G + 0.500B) + 128 (3);

[0081] Extract the image g from the current monitored video stream. First, traverse all the points in g, and pass the Y, Cr, and Cb values of each point into the mynew array. Similarly, store the Y, Cr, and Cb values at the corresponding position in bk into the myold array. Calculate the absolute value of the difference between the three corresponding elements of the two arrays respectively. If all three differences are less than their respective thresholds of 40, 15, and 15, it indicates that the information of the two is similar, and the corresponding points can be marked as the background, that is, mark the corresponding position in fo as 0, which is the normal value; otherwise, store the Y, Cr, and Cb values of the dynamic background frame bk1 into the myold array, and make the same comparison with mynew as bk. If it is determined to be the background, also mark the corresponding position in fo as 0, otherwise mark it as 255, which is the abnormal value.

[0082] Traverse all the unmarked points in the current frame (the points with 0 in the fo marking array), and update the corresponding positions in bk1 with their Y, Cr, and Cb parameters, so that the differential judgment can adapt to the slowly changing background.

[0083] Perform opening and closing operations on fo, that is, erosion, dilation, dilation, and erosion operations, and select a 3×3 template for the opening and closing operations.

[0084] Perform connectivity detection on fo to obtain the left and right connected regions. Remove the connected regions with fewer than 600 or more than 40,000 points, and then verify the aspect ratio. Remove the connected regions with an aspect ratio greater than 5 or less than 0.2. Record the number of all connected regions into rect_number. Each connected region has four coordinate elements: the upper left corner coordinates (x1, y1), the lower right corner coordinates (x2, y2). Store the coordinates of all connected regions into the cord array, with a total of 4*rect_number elements. In Example 1, the number of connected regions rect_number returned is 1, and the value of the cord array is (565, 516, 596, 567); in Example 2, the rect_number returned is 2, and the value of the cord array is (210, 448, 246, 491, 621, 472, 654, 522). The recorded parameters are shown in Table 1, which are the load constraints for all connected regions.

[0085]

[0086] Table 1 The ratio of the number of marked points in the connected region to the size of the rectangular frame

[0087] Perform texture feature verification on the detected connected regions. Take out the central 1 / 3 part of the region, as shown in formulas (4) to (7). For the region determined by x_1, y_1, h_2, and y_2, calculate the texture map for the points in this region of image g: the average value of the absolute values of the difference between this point and its left point, and the absolute value of the difference between this point and its upper point; after obtaining the texture map, calculate the eigenvalue for this texture map: that is, sort the average values of the four regions divided by the texture map, with a total of 4*3*2*1 = 24 possibilities, obtaining 24 different eigenvalues, and each value corresponds to a unique sorting situation.

[0088] Similarly, for the regions determined by x_1, y_1, h_2, and y_2, calculate the eigenvalues for the background images bk and bk1 in the same way. If the eigenvalue of this region of the image g to be detected is the same as any one of the two backgrounds, then discard this connected region to obtain more reliable connected region coordinates cord1;

[0089] x_1 = x1+(x2 - x1) / 3 (4);

[0090] y_1 = y1+(y2 - y1) / 3 (5);

[0091] x_2 = x2-(x2 - x1) / 3 (6);

[0092] y_2 = y2-(y2 - y1) / 3 (7);

[0093] Dwell time and occlusion processing. Examine all position points in image g. If a point is not in any connected region of cord1 (that is, this point is not marked as a suspected point in image g), then analyze the value of fg at this position. If the value of fg at this position = 0, the value of fo1 at this position is recorded as 0, that is, a normal point; otherwise, the value of fg is first -1. If fg <= M after subtracting 1, it indicates that the occlusion frames have exceeded C, and the value of fo1 at this position is also recorded as 0. Otherwise, the occlusion time has not arrived, and the value of fo1 at this position is recorded as 255, that is, a foreign object point.

[0094] If a point is in a certain connected region of cord1 (that is, this point is marked as a suspected point in image g), then judge the value of fg at this point. If the value of fg >= N, the value of fo1 at this position is recorded as 255. Otherwise, the value of fg is first +1. If fg = N after increasing, the value of fo1 at this position is also recorded as 255. Otherwise, the value of fo1 at this position is recorded as 0;

[0095] fo1 records the finally determined target region. Perform connected region detection on it to obtain the rectangular frames of all connected regions. Each rectangular frame represents a remaining object target, and report this detection result, as Figure 4 shown, which are the detection results of two scenarios respectively. Then, perform the detection of the next frame.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Those of ordinary skill in the art should understand that any modifications or equivalent replacements made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions should be covered within the scope of the claims of the present invention.

Claims

1. A foreign object detection method for substation security monitoring videos, characterized in that, It includes the following steps: S1. Obtain three process reference diagrams fo, fo1, and fg with the same image size as the historical video stream, and set the initial attribute values of each point in fo, fo1, and fg to 0; then screen out two frames of images from the images of the historical video stream and use them as the static background frame bk and the dynamic background frame bk1 respectively; S2. After extracting a frame of image g to be detected from the video stream to be measured according to a preset frame extraction method; compare each point in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively. If there is a foreground object point in g, record the attribute value of the corresponding point in fo as an abnormal value; If there is no point in fo with an abnormal attribute value, go to S3; otherwise, go to S4; S3. Use the image g as the updated dynamic background frame bk1 and return to S2; S4. Screen out the points in fo with abnormal attribute values according to a preset abnormal point screening method, and use the points after screening as suspected points; S5. Add 1 to the attribute value of the point in fg corresponding to each suspected point, and record the number of times each point is marked as a suspected point through the attribute value of each point in fg; S6. Statistically calculate the growth value A of the attribute value of each point in fg within a preset time period, and judge whether each point is a foreign object point according to the growth value A of each point by a preset foreign object judgment method. If so, record the attribute value of the corresponding point in fo1 as a foreign object value; S7. Analyze whether there is a point in fo1 with a foreign object value. If not, return to S2; if so, after reporting the foreign object, return to S2.

2. The foreign object detection method for the substation safety monitoring video according to claim 1, wherein: In S2, when comparing the points in the image g with the corresponding points in the static background frame bk and the dynamic background frame bk1 respectively, first calculate the difference between the pixels of the points in the image g and the corresponding pixels in the static background frame bk; if the absolute value of the difference is greater than or equal to the set value, then calculate the difference between the pixels of the point and the corresponding pixel in the dynamic background frame bk1. If the absolute value of the difference is still greater than the set value, then mark the point as a foreground object point.

3. The foreign object detection method for the substation safety monitoring video according to claim 1, wherein: S4 includes: S41. Perform an opening operation on fo to obtain fo01; S42. Obtain the connected regions of the points in fo01 with abnormal attribute values, and perform a preset connected region inspection and screening. Record the remaining connected regions after inspection and screening as effective connected regions; S43. Perform a preset texture feature verification on the regions in the image g corresponding to each effective connected region, and record the points in the effective connected region corresponding to the remaining regions after passing the texture feature verification as suspected points.

4. The foreign object detection method for the substation safety monitoring video according to claim 3, characterized in that: In S41, after performing an opening operation and a closing operation on fo, obtain fo01.

5. The foreign object detection method for the substation safety monitoring video according to claim 3, wherein: In S42, the preset connected region inspection and screening includes: for each connected region in fo01, record the coordinates of all points with abnormal attribute values in the connected region, and calculate the number of points with abnormal attribute values in the connected region. If the number is less than the preset low threshold or greater than the preset high threshold, then screen out the connected region.

6. The foreign object detection method for the substation safety monitoring video according to claim 5, wherein: In S42, the preset connected region inspection and screening further includes: recording the vertex coordinates of the circumscribed rectangle of each connected region, and after calculating the aspect ratio of the circumscribed rectangle based on the vertex coordinates of the circumscribed rectangle of the connected region, screening out the connected regions with an aspect ratio greater than the preset ratio.

7. The foreign object detection method for the substation safety monitoring video according to claim 6, wherein: In S42, the preset connected region inspection and screening further includes: after calculating the area of the circumscribed rectangle based on the vertex coordinates of the circumscribed rectangle of the connected region, calculating the ratio of the area occupied by the points with abnormal attribute values in the connected region to the area of the circumscribed rectangle of the connected region, and performing a shape constraint verification on the detection target based on this ratio and the aspect ratio of the circumscribed rectangle of the connected region. If the result of the constraint verification is not a foreign object, the corresponding connected region is screened out.

8. The foreign object detection method for the substation safety monitoring video according to claim 3, characterized in that: In S43, the preset texture feature verification includes: S431. Respectively extract the texture maps of the regions corresponding to each effective connected region in the image g, the static background frame bk, and the dynamic background frame bk1. When extracting the texture map, the texture of a point is the average value of the absolute value of the difference between this point and its left point and the absolute value of the difference between this point and its upper point. S432. Perform feature extraction on each texture map. Draw a horizontal dividing line and a vertical dividing line from the center of the texture map to divide the region into four sub-regions of equal size, and record the region numbers of the four sub-regions as 1, 2, 3, and 4 respectively. After summing the texture map values in each sub-region respectively, sort them in descending order. Then, obtain the feature value of the texture map according to the preset feature value calculation method. The feature value calculation method is: the region number ranked first * 1000 + the region number ranked second * 100 + the region number ranked third * 10 + the region number ranked fourth * 1. S433. Analyze the feature values of the texture maps. If the feature value of a texture map in the image g is the same as the feature value of the corresponding texture map in the static background frame bk or the dynamic background frame bk1, the corresponding effective detection region in fo01 is screened out.

9. The foreign object detection method for the substation safety monitoring video according to claim 8, wherein: In S431, when extracting the texture maps of the regions corresponding to each effective connected region in the image g, the static background frame bk, or the dynamic background frame bk1, only extract the texture map of the central position region at the exact center. The length of the central position region is 1 / n of the corresponding effective connected region, and the width of the central position region is 1 / n of the corresponding effective connected region.

10. The foreign object detection method for the substation safety monitoring video according to claim 1, characterized in that: In S6, the preset foreign object judgment method includes: judging whether each point in the image g is a suspected point in fo01; If it is a suspected point, judge whether the growth value A of this point plus 1 is greater than or equal to N. If so, this point is regarded as a foreign object point; if not, this point is regarded as a normal point. Here, N is the preset first threshold. If it is not a suspected point, judge whether the growth value A of this point is 0. If it is 0, this point is regarded as a normal point; if it is not 0, subtract 1 from the growth value A of this point and then judge whether it is greater than M. If it is greater than M and the detection result of this point in the previous frame is a suspected point, this point is regarded as a foreign object point; if it is less than or equal to M or the detection result of this point in the previous frame is not a suspected point, this point is regarded as a normal point. Here, M is the preset second threshold, and M < N.

Images