Method and System for Detecting the Movement Trajectory of Objects in a Substation Based on Computer Vision

Abstract

The present invention discloses a method and system for detecting the movement trajectory of an object in a substation based on computer vision, including: performing foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene; extracting the feature values of the foreign objects from the foreign object pictures; recognizing the feature values of the foreign objects to obtain the type information of the foreign objects; if the type information of the foreign objects meets the preset storage conditions, inputting the type information of the foreign objects into the database; obtaining the movement data matching the target foreign object to be detected from the database, and obtaining the historical movement trajectory of the target foreign object according to the position information and time information in the movement data of the target foreign object; determining the movement trajectory of the target foreign object after the current monitoring scene according to the position information of the target foreign object in the current monitoring scene and the historical movement trajectory. The present invention can accurately predict the movement trajectory of a foreign object after the current monitoring scene, providing technical support for the trajectory recognition of an object.

Description

Technical Field

[0001] The present invention relates to the field of object motion detection, and particularly to a method and system for detecting the motion trajectory of an object in a substation based on computer vision. Background Art

[0002] Typical objects in a substation have complex structures, a large variety and quantity of equipment, and wide dispersion. Traditional methods for identifying the motion trajectory of targets have problems such as low recognition accuracy, poor robustness to diverse environments, lack of pertinence, and time consumption. Currently, due to actual technical level limitations, when typical foreign objects invade a substation, it is impossible to quickly and accurately predict the motion trajectory of the foreign objects after the current monitoring scene.

[0003] Therefore, a method and system for detecting the motion trajectory of an object in a substation based on computer vision are needed, which can improve the problem of being unable to quickly and accurately predict the motion trajectory of foreign objects after the current monitoring scene. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to overcome the defects in the prior art, and provide a method and system for detecting the motion trajectory of an object in a substation based on computer vision, which can accurately predict the motion trajectory of foreign objects after the current monitoring scene, and provide reliable technical support for object recognition and motion trajectory prediction in the substation.

[0005] The method for detecting the motion trajectory of an object in a substation based on computer vision according to the present invention includes:

[0006] Performing foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene;

[0007] Extracting the feature values of the foreign objects from the foreign object pictures; performing recognition on the feature values of the foreign objects to obtain the type information of the foreign objects;

[0008] Judging whether the type information of the foreign objects meets the preset storage conditions. If so, entering the type information of the foreign objects into the database; if not, not performing any processing;

[0009] Obtaining motion data matching the target foreign object to be detected from the database, and obtaining the historical motion trajectory of the target foreign object according to the position information and time information in the motion data of the target foreign object;

[0010] Determining the motion trajectory of the target foreign object after the current monitoring scene according to the position information of the target foreign object in the current monitoring scene and the historical motion trajectory.

[0011] Further, performing foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene specifically includes:

[0012] S11. Slice the video data at a specified frame interval to form n pictures in time sequence;

[0013] S12. Denote the nth picture and the (n - 1)th picture as F n and F n-1 respectively. Denote the gray values of the corresponding pixel points of the two pictures as F n (x, y) and F n-1 (x, y). Subtract the gray values of the corresponding pixel points of the two images and take the absolute value to obtain the difference image D n ;

[0014] S13. Classify the gray value of any pixel point in the difference image D n into one of the two values 0 or 255 to obtain the binary image R n ';

[0015] S14. Perform connectivity analysis on the binary image R n ' to obtain the image R n containing the complete moving foreign object;

[0016] S15. Label the m foreign objects in the image R n and mark the area range where the foreign objects are located: Rang n (x n1 , y n1 , x n2 , y n2 ); where m is an integer representing the total number of foreign objects recognized in the scene picture; in Rang n (x n1 , y n1 , x n2 , y n2 ), n ∈ (1, 2,..., m), (x n1 , y n1 ) represents the upper left coordinate of the area where the nth foreign object is located, and (x n2 , y n2 ) represents the lower right coordinate of the area where the nth foreign object is located;

[0017] S16. Perform image segmentation on the image areas of the m foreign objects respectively according to the area range Rang n (x n1 , y n1 , x n2 , y n2 ) to obtain m pictures corresponding to the m foreign objects.

[0018] Furthermore, extract the feature values of the foreign objects from the foreign object pictures, specifically including:

[0019] Perform feature enhancement on the foreign object image to obtain an enhanced foreign object image;

[0020] Extract texture features, edge features, and grayscale mean features from the enhanced foreign object image to obtain a feature point dataset corresponding to the respective features, and use the feature point dataset as the feature value of the foreign object.

[0021] Furthermore, identify the type information of the foreign object from the feature value of the foreign object, specifically including: performing iterative operations on the feature point dataset through the RANSAC algorithm and finding the optimal parameters to minimize the cost function to obtain the type information corresponding to each foreign object; wherein, the type information includes at least one of a kite, a bird's nest, a hydrogen balloon, a tree branch, and a disposable plastic product.

[0022] Furthermore, determine whether the type information of the foreign object meets the preset storage conditions, specifically including:

[0023] Perform screening and verification on the type information of the foreign object through a preset screening strategy;

[0024] When the type information of the foreign object does not exist in the database and there is no feature anomaly in the type information of the foreign object determined based on the preset screening strategy, it is determined that the type information of the foreign object meets the preset storage conditions; wherein, the feature anomaly includes at least one of color or shape anomaly, abnormal time format being recorded, and color or shape error exceeding a preset threshold.

[0025] Furthermore, obtain the historical movement trajectory of the target foreign object based on the position information and time information in the movement data of the target foreign object, specifically including:

[0026] Arrange the position information of the target foreign object in time series according to the position information and time information in the movement data of the target foreign object to obtain the position information corresponding to the relevant time period;

[0027] Based on the position information corresponding to the relevant time period, determine the historical movement trajectory of the target foreign object corresponding to the relevant time period.

[0028] Furthermore, determine the movement trajectory of the target foreign object after the current monitoring scenario based on the position information of the target foreign object in the current monitoring scenario and the historical movement trajectory, specifically including:

[0029] Analyze the historical trajectory of the target foreign object to obtain the route selection of the target foreign object from one position to the next position and the selection probability wherein, represents selecting route i, represents the probability that the target foreign object selects route ;

[0030] According to the frequency and number of foreign objects appearing in the current scene, the foreign objects are divided into periodic foreign objects and non-periodic foreign objects, and the movement trajectory of the target foreign object after the current monitoring scene is obtained; wherein, the foreign objects include the target foreign object and other foreign objects;

[0031] The probability p of selecting line i in the movement trajectory of the target foreign object after the current monitoring scene i :

[0032]

[0033] wherein, w t represents the weight set as the probability w, c represents the weight set as the probability p ci and represents the probability that other foreign objects select route i.

[0034] Furthermore, before extracting the feature values of the foreign objects from the foreign object pictures, it further includes:

[0035] Determine the membership function corresponding to the foreign object flow density state in the monitoring scene according to the number of foreign object pictures obtained within a preset time period;

[0036] Determine the sending period of the foreign object pictures according to the membership function, and send the foreign object pictures to the monitoring device at the sending period.

[0037] Furthermore, determining the membership function corresponding to the foreign object flow density state in the monitoring scene specifically includes:

[0038] Establish a comprehensive evaluation factor set U=(u1, u2, u3) according to the influencing factors of the foreign object flow, where u1 represents the foreign object density in the scene, u2 represents the current residence time, and u3 represents the foreign object movement time in the current scene;

[0039] Set the evaluation set V=(v1, v2, v3, v4) of the foreign object flow density state, where v1 represents not dense, v2 represents ordinary dense, v3 represents relatively dense, and v4 represents very dense;

[0040] Let the membership degree of the i-th element in the factor set U to the j-th element in the evaluation set V be r ij , then the membership function corresponding to each foreign object flow density state is:

[0041] The membership function r 11 (m) for not dense:

[0042] The membership function r 12 (m) for ordinary dense:

[0043] The relatively dense membership function r 13 (m):

[0044] The very dense membership function r 14 (m):

[0045] Where m represents the total number of foreign objects in the scene.

[0046] A computer vision-based object motion trajectory detection system in a substation, comprising: a monitoring device, an edge server, and a data management server;

[0047] The monitoring device is used to identify foreign objects from the collected scene video data to obtain foreign object pictures in the substation monitoring scene;

[0048] The edge server is used to extract the feature values of the foreign objects from the foreign object pictures; identify the feature values of the foreign objects to obtain the type information of the foreign objects;

[0049] The data management server is used to determine whether the type information of the foreign object meets the preset storage conditions. If so, it enters the type information of the foreign object into the database; if not, it does not process; obtains the motion data matching the target foreign object to be detected from the database, and based on the position information and time information in the motion data of the target foreign object, obtains the historical motion trajectory of the target foreign object; determines the motion trajectory of the target foreign object after the current monitoring scene according to the position information of the target foreign object in the current monitoring scene and the historical motion trajectory.

[0050] The beneficial effects of the present invention are: A computer vision-based object motion trajectory detection method and system disclosed by the present invention first extracts pictures with abnormal values in the monitoring scene from the scene video data, then extracts the feature values of foreign objects from the abnormal value pictures, and identifies the type information of foreign objects based on the feature values, and when meeting the corresponding storage conditions, enters the foreign object information into the database for convenient later indexing and calling. When it is necessary to detect the trajectory of the target foreign object, the historical motion trajectory of the target foreign object can be obtained from the database first, and then combined with the position information of the target foreign object in the current monitoring scene, the motion trajectory after the current monitoring scene is predicted. Trajectory prediction based on the historical motion trajectory is beneficial to improving the accuracy of trajectory prediction and provides a reliable technical support for the identification of typical objects and the prediction of motion trajectories in the substation. Description of the Drawings

[0051] The present invention will be further described below in conjunction with the drawings and embodiments:

[0052] Figure 1Schematic diagram of the detection method process of the present invention;

[0053] Figure 2 Schematic diagram of the principle of the foreign object recognition algorithm of the present invention;

[0054] Figure 3 Schematic diagram of the communication connection structure of the detection system of the present invention. Specific implementation mode

[0055] The following further describes the present invention in conjunction with the attached drawings of the specification, as shown in the figure:

[0056] The method for detecting the movement trajectory of an object in a substation based on computer vision of the present invention includes:

[0057] S1. Perform foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene;

[0058] S2. Extract the characteristic values of the foreign object from the foreign object pictures; identify the characteristic values of the foreign object to obtain the type information of the foreign object;

[0059] S3. Judge whether the type information of the foreign object meets the preset storage conditions. If so, enter the type information of the foreign object into the database; if not, do not process it;

[0060] S4. Obtain the movement data matching the target foreign object to be detected from the database, and obtain the historical movement trajectory of the target foreign object according to the position information and time information in the movement data of the target foreign object;

[0061] S5. Determine the movement trajectory of the target foreign object after the current monitoring scene according to the position information of the target foreign object in the current monitoring scene and the historical movement trajectory.

[0062] In this embodiment, the monitoring device performs foreign object recognition on the collected scene video data through a preset target recognition algorithm to obtain foreign object pictures in the substation monitoring scene, specifically including:

[0063] S11. Slice the video data at a specified frame interval to form n pictures in time sequence; where the specified frame interval refers to the number of image frames at intervals, which can be flexibly selected according to actual situations.

[0064] S12. As Figure 2 shown, record the nth picture and the (n - 1)th picture as F n and F n-1 , record the gray values of the corresponding pixel points of the two pictures as F n (x, y) and F n-1 (x, y), subtract the gray values of the corresponding pixel points of the two images and take the absolute value to obtain the difference image Dn ;

[0065] Further, obtain:

[0066] D n (x, y) = |F n (x, y) - F n-l (x, y)| (i - i)

[0067] In formula (1 - 1), D n (x, y) represents the gray value of each pixel point of the difference image D n . x and y represent the coordinate positions of the pixel point in the picture, and n represents the sequence of the icon;

[0068] S13. Set a threshold T to perform threshold processing on the difference image. To achieve a better effect, T = 0.05 can be taken, and each pixel point is binarized according to the following formula (1 - 2), that is, according to formula (1 - 2), the gray value of any pixel point is classified into one of the two values 0 or 255 to obtain the binarized image R n '. Among them, the point with a gray value of 255 is the foreground (moving object) point, usually representing a moving object; the point with a gray value of 0 is the background point, usually representing a stationary object.

[0069]

[0070] S14. Perform connectivity analysis on the binarized image R n ' to obtain an image R n containing a complete moving foreign object; among them, the connectivity analysis adopts existing connectivity analysis means, which will not be elaborated here; through connectivity analysis, it can be determined whether the shape of the object in the connected region in the picture is a foreign object, so that the positions of each foreign object in the picture can be marked.

[0071] S15. Label the m foreign objects in the image R n and mark the range of the area where the foreign object is located: Rang n (x n1 , y n1 , x n2 , y n2 ); where m is an integer representing the total number of foreign objects identified in the scene picture; in Rang n (x n1 , y n1 , x n2 , y n2 ), n ∈ (1, 2,..., m), (x n1 , y n1 ) represents the upper left corner coordinates of the area where the nth foreign object is located, (x n2, y n2 ) represents the lower right corner coordinates of the area where the nth foreign object is located;

[0072] S16. According to the range Rang of the area where the foreign object is located n (x n1 , y n1 , x n2 , y n2 ) perform image segmentation on the image areas of m foreign objects respectively to obtain m pictures corresponding to the m foreign objects.

[0073] In this embodiment, when the edge server receives the foreign object picture sent by the monitoring device, it extracts the feature values of the foreign object from the foreign object picture, specifically including:

[0074] Perform feature enhancement on the foreign object picture to obtain an enhanced foreign object picture; among them, the existing defogging algorithm or histogram equalization method can be used to enhance the picture p to obtain the picture p';

[0075] Extract texture features, edge features, and gray mean features from the enhanced foreign object picture to obtain a feature point data set corresponding to the corresponding features, and use the feature point data set as the feature value of the foreign object. That is, a first feature point data set for the foreign object contour, a second feature point data set for the foreign object texture, and a third feature point data set for the foreign object gray value can be obtained respectively, and the obtained feature point data set can be used as the feature value of the corresponding feature of the foreign object.

[0076] In this embodiment, the edge server identifies the feature values of the foreign object through a preset feature recognition algorithm to obtain the type information of the foreign object, specifically including: performing iterative operations on the feature point data set through the RANSAC algorithm and finding the optimal parameters to minimize the cost function to obtain the type information corresponding to each foreign object; among them, the type information includes at least one of kite, bird's nest, hydrogen balloon, branch, and disposable plastic products.

[0077] The cost function is as follows:

[0078]

[0079] Among them, C refers to the cost function; n refers to the total number of pictures; x and y represent the corner position coordinates of the target image; x' and y' represent the corner position coordinates of the scene image; h refers to the parameter in the homography matrix;

[0080] The homography matrix is:

[0081]

[0082] In the homography matrix, s represents the scale factor.

[0083] Generally, let h 33 = 1 for normalization. Generally speaking, the homography matrix has 8 parameters, and at least 4 pairs of matching points are required to solve the equation. Using the RANSAC algorithm, the optimal parameters are continuously iteratively searched to minimize the cost function C. The function of the homography matrix is to correct the image and perform perspective transformation. For example, to convert an obliquely taken image into a front view, a relationship mapping needs to be established between each pixel point in the original image and the corresponding pixel point after conversion. The parameter h represents the relationship mapping parameter between the original pixel point coordinates (x, y) and the new pixel point coordinates (x′, y′).

[0084] Among them, the steps of the RANSAC algorithm are as follows:

[0085] Step A: Randomly select 4 groups of samples from the set S to be matched, and ensure that the 4 samples in the same group are not collinear. Then, use the 4 groups of samples to estimate the homography matrix H, denoted as model M;

[0086] Step B: Calculate the projection error between the remaining sample data in the matching set S and the model M. If the projection error is less than the specified threshold, add it to the "inlier" set I; otherwise, mark it as an "outlier". The specified threshold can be flexibly set according to the actual situation;

[0087] Step C: If the number of samples in the current set I is greater than the optimal set I_best, update I_best;

[0088] Step D: If the number of iterations exceeds k0 times, exit the algorithm; otherwise, re-iterate the above steps;

[0089] The RANSAC algorithm estimates the number of iterations k0 using the following formula:

[0090]

[0091] In formula (5-3), m represents the minimum number of samples required to estimate the model, p represents the probability that the algorithm can give the optimal model, w represents the ratio of "inliers", and w is usually continuously updated in the iterative algorithm.

[0092] In this embodiment, when the data management server receives the type information of the foreign object, it determines whether the type information of the foreign object meets the preset storage conditions, specifically including:

[0093] Screen and verify the type information of the foreign object through a preset screening strategy;

[0094] When the type information of the foreign object does not exist in the database and there is no feature anomaly in the type information of the foreign object determined based on the preset screening strategy, it is determined that the type information of the foreign object meets the preset warehousing conditions; wherein, the feature anomaly includes at least one of color or shape anomaly, abnormal time format recorded, and color or shape error exceeding the preset threshold.

[0095] The preset screening strategy can be flexibly set according to the actual situation. For example, the preset screening strategy can be used to detect the data characteristics of each data group in the foreign object information, and eliminate some data groups with obvious data anomalies. For example, if the color or shape of the foreign object is different from the normal one, it indicates that there is an anomaly in the foreign object; if the time format in which the data is recorded is different from the normal time format, it indicates an abnormal time format; due to environmental factors or other factors, the color or shape of the foreign object has an error compared with the color or shape under normal environmental factors. At this time, it is necessary to screen the error data and filter the parameters whose error exceeds the preset threshold, where the preset threshold can be flexibly set according to the actual situation.

[0096] Among the data groups of the foreign object information, the error data screening method can be as follows:

[0097] Considering the influence of environmental factors and the accuracy of image processing, some parameters in the collected data may change. Therefore, it is necessary to screen some possible error data. The screening process can be as follows:

[0098] Set a similarity variable S, similarity threshold S max = 90%, and a credibility variable B, credibility threshold B min = 5%;

[0099] Traverse the data groups in the sample database. When it is found that the similarity of two groups of data or multiple groups of data exceeds the threshold S max , judge the credibility of the data;

[0100] Taking n data groups data1 to data max whose similarity exceeds the threshold S n as an example, conduct a credibility judgment. Respectively obtain the occurrence times labels of the two groups of data from the database Use formula (7-1) to calculate the credibility of each data group:

[0101]

[0102] Compare the credibility of each group of data with the credibility threshold B min . If the feasibility is lower than the threshold, judge that this group of data is error data and eliminate it from the database.

[0103] In this embodiment, in step S4, the target foreign object to be detected can be flexibly determined according to the actual situation. For example, the user can determine the target foreign object by inputting the type of foreign object to be detected, or specify a foreign object as the target foreign object from the captured scene video data, or determine the target foreign object according to other situations.

[0104] When it is necessary to detect the target foreign object, the data management server can extract the corresponding identification information of the target foreign object. For example, the circumscribed rectangle of the target foreign object. Then, based on the identification information, the motion data matching the identification information is filtered out from the database. For example, in the database, the feature information of each type of foreign object is associated with the motion data of the foreign object. In this way, the motion data corresponding to the foreign object with the same foreign object feature information as the target foreign object can be used as the motion data of the target foreign object.

[0105] According to the position information and time information in the motion data of the target foreign object, the historical motion trajectory of the target foreign object is obtained, specifically including:

[0106] According to the position information and time information in the motion data of the target foreign object, the position information of the target foreign object is arranged in time series to obtain the position information corresponding to the relevant time period;

[0107] Based on the position information corresponding to the relevant time period, the historical motion trajectory of the target foreign object corresponding to the relevant time period is determined.

[0108] That is, the data management server arranges the position information of the foreign object in time series according to the time series in the driving data, combined with the label of the node position attached to the data, and constructs a data group indexed by the node position and time series where xi represents the i-th trajectory of the foreign object, is an array related to the time series, represents the j-th position passed by the foreign object in the i-th historical trajectory. In this way, the reconstruction of the historical trajectory of the foreign object can be realized, and the historical motion trajectory of the target foreign object corresponding to the relevant time period can be obtained.

[0109] In this embodiment, in step S5, according to the position information of the target foreign object in the current monitoring scene and the historical motion trajectory, the motion trajectory of the target foreign object after the current monitoring scene is determined, specifically including:

[0110] The data management server can analyze the historical trajectory of the target foreign object to obtain the position transfer selection and selection probability of the target foreign object at each node position, that is, the selection scheme F from the position node a to the next position node (b, c, d, e,...) a (f ab ,f ac ,f ad ,fae ,...) and the corresponding selection probability P a (p ab , p ac , p ad , p ae ,...), where the specific value of the probability is obtained by dividing the number of trajectories containing this section of the trajectory in the historical trajectory database by the total number of all trajectories passing through this node. For a region composed of n nodes, a probability matrix F = [p ij n×n .

[0111] Based on the probability matrix, the data management server establishes a first-order Markov model, that is, the position of the target foreign object at a certain moment is only related to the previous position. Therefore, in the matrix F, only when the positions of two nodes are adjacent, the transition probability is not 0. In order to predict the probability of trajectories with multiple nodes and non-adjacent starting and ending positions, it is necessary to calculate the transition probabilities of all possible solutions. Suppose the probabilities of the foreign object passing through nodes a, b, c, and d in sequence are p abcd , then the following formula (9-1) holds:

[0112] p abcd = p ab ·p bc ·p cd (9-1)

[0113] Meanwhile, when there are two paths from node a to node d, namely a->b->c->d and a->c->d, then there is:

[0114] p ad = p abcd + p acd (9-2)

[0115] Based on the above analysis, the route selection of the target foreign object from a certain position to the next position and the selection probability can be obtained, where represents the selected route i, represents the probability that the target foreign object selects route ; in addition, by reconstructing the historical trajectories of other foreign objects at the intersection where the target foreign object is located, the trajectory set of recently passing foreign objects in this scenario can be obtained:

[0116] where x i represents the trajectory of the i-th foreign object in this scenario, is an array related to the time series, represents the j-th position passed by the i-th foreign object in the historical trajectory in this scenario;

[0117] ​According to the frequency and number of foreign objects appearing in the current scenario, the foreign objects are divided into periodic foreign objects and non-periodic foreign objects, and the movement trajectory of the target foreign object after the current monitoring scenario is obtained; wherein, the foreign objects include the target foreign object and other foreign objects;

[0118] The probability p of selecting route i in the movement trajectory of the target foreign object after the current monitoring scenario i :

[0119]

[0120] wherein, w t represents the weight set as the probability w c represents the weight set as the probability p ci , represents the probability that other foreign objects select route i. The weights w t and w c can be set separately according to different situations where the target foreign object belongs to periodic foreign objects or non-periodic foreign objects.

[0121] Based on the above design, the monitoring device is used to collect data on foreign objects passing through each road node, and the data is stored in the data management server through the edge server. A model is established to screen the acquired data, eliminate incorrect or duplicate data, and realize the reorganization, packaging, and marking of the data, generating a database of cloud foreign object information. When predicting the trajectory of the foreign object target on the road, the trajectory of the foreign object registered in the database and the recorded foreign object information can be reconstructed. Then, combined with the historical driving trajectory of the target foreign object itself and the driving trajectory trend of other foreign objects at the current intersection, the trajectory of the target foreign object after the current intersection can be comprehensively predicted. In this way, it is beneficial to improve the accuracy and reliability of trajectory prediction.

[0122] In this embodiment, before the edge server receives the foreign object picture sent by the monitoring device, that is, before the edge server extracts the feature value of the foreign object from the foreign object picture, it further includes:

[0123] The monitoring device determines the membership function corresponding to the foreign object flow density state in the monitoring scenario according to the number of foreign object pictures obtained within the preset time period;

[0124] The monitoring device determines the sending period of the foreign object picture according to the membership function, and sends the foreign object picture to the monitoring device at the sending period.

[0125] Among them, determining the membership function corresponding to the foreign object flow density state in the monitoring scenario specifically includes:

[0126] A comprehensive evaluation factor set U = (u1, u2, u3) is established according to the influencing factors of foreign object flow rate. Among them, u1 represents the density of foreign objects in the scene, u2 represents the current residence time, and u3 represents the movement time of foreign objects in the current scene;

[0127] Set the evaluation set V = (v1, v2, v3, v4) for the intensive state of foreign object flow rate. Among them, v1 represents not intensive, v2 represents ordinary intensive, v3 represents relatively intensive, and v4 represents very intensive;

[0128] Let the membership degree of the i-th element in the factor set U to the j-th element in the evaluation set V be r ij , then the membership degree functions corresponding to each intensive state of foreign object flow rate are as follows:

[0129] The membership degree function r 11 (m):

[0130] The membership degree function r 12 (m):

[0131] The membership degree function r 13 (m):

[0132] The membership degree function r 14 (m):

[0133] Among them, m represents the total number of foreign objects in the scene.

[0134] According to the membership degree function, determine the sending period for sending the foreign object picture, specifically including:

[0135] Based on the membership degree functions r 11 (m), r 12 (m), r 13 (m) and r 14 (m), determine the evaluation sets R1, R2, and R3 corresponding to the influencing factors u1, u2, and u3 respectively;

[0136] According to the above membership degree function formula, finally obtain the evaluation result R1 = (r 11 , r 12 , r 13 , r 14 ) of the influencing factor u1. Further, based on the influencing factors u2 and u3 in the influencing factor set U = (u1, u2, u3) and the above membership degree function formula, a total of 3 single-factor evaluation sets R1, R2, and R3 can be obtained, and finally form a 3*4 fuzzy comprehensive evaluation matrix R 3*4 .

[0137] Next, the monitoring device determines the weight set W (w1, w2, w3), and the weights are used to reflect the influence degree of each factor on the evaluation object. In this embodiment, the monitoring device can use the average weight method to determine the weights of the influencing factors of the foreign object flow state, and can take

[0138] The monitoring device establishes a comprehensive evaluation model, and performs a fuzzy evaluation on the traffic flow state through the weight vector W and the fuzzy comprehensive evaluation R 3*4 matrix. Set the evaluation result as C0, then there is:

[0139] C0 = W * R = (c1, c2, c3, c4)(3 - 5)

[0140] In formula (3 - 5), W refers to the weight vector, and R is the fuzzy comprehensive evaluation matrix R 3*4 , c i (i = 1, 2, 3, 4) represents the probability corresponding to the i-th foreign object flow state, and

[0141] Finally, the monitoring device autonomously changes the sending period of data transmission according to the inferred traffic flow state. For example, the basic transmission period T = 2 hours or other durations can be set. To better adapt to the changing situation of the scene traffic flow, according to the above result C0, a comprehensive evaluation index k is set,

[0142] Let It can be known that when the foreign object flow state is denser, the value of k is larger, and the data collected by the data acquisition unit in the same time is also more. Therefore, the data transmission period of the device should be reduced accordingly, and a new data transmission period parameter T * is used to replace the original period T, and T * satisfies the relational formula (3 - 6)

[0143] T * = (T / k)(3 - 6)

[0144] The present invention also relates to a system for detecting the movement trajectory of objects in a substation based on computer vision. The system corresponds to the above-mentioned method for detecting the movement trajectory of objects in a substation, and can be understood as a system for implementing the above method. As Figure 3 shown, it includes: a monitoring device, an edge server, and a data management server;

[0145] The monitoring device is used to identify foreign objects in the collected scene video data to obtain foreign object pictures in the substation monitoring scene;

[0146] The edge server is configured to extract the feature values of the foreign object from the foreign object picture, and identify the feature values of the foreign object to obtain the type information of the foreign object.

[0147] The data management server is configured to determine whether the type information of the foreign object meets the preset storage condition. If so, it enters the type information of the foreign object into the database; if not, it does not perform any processing. It obtains the motion data matching the target foreign object to be measured from the database, and obtains the historical motion trajectory of the target foreign object according to the position information and time information in the motion data of the target foreign object. It determines the motion trajectory of the target foreign object after the current monitoring scenario according to the position information of the target foreign object in the current monitoring scenario and the historical motion trajectory.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for detecting the movement trajectory of objects in a substation based on computer vision, characterized in that: Including: Performing foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene; Extracting the characteristic values of the foreign objects from the foreign object pictures; Identifying the characteristic values of the foreign objects to obtain the type information of the foreign objects; Before extracting the characteristic values of the foreign objects from the foreign object pictures, it further includes: Determining the membership function corresponding to the foreign object flow density state in the monitoring scene according to the number of foreign object pictures obtained within a preset time period; wherein, a comprehensive evaluation factor set U=(u1, u2, u3) is established according to the influencing factors of the foreign object flow, where u1 represents the foreign object density in the scene, u2 represents the current residence time, and u3 represents the foreign object movement time in the current scene; Determining the sending period of the foreign object pictures according to the membership function, and sending the foreign object pictures to the monitoring device at the sending period; Judging whether the type information of the foreign object meets the preset storage condition. If so, entering the type information of the foreign object into the database; if not, no processing is performed; Obtaining the motion data matching the target foreign object to be measured from the database, and obtaining the historical motion trajectory of the target foreign object according to the position information and time information in the motion data of the target foreign object; Determining the motion trajectory of the target foreign object after the current monitoring scene according to the position information of the target foreign object in the current monitoring scene and the historical motion trajectory, specifically including: Analyze the historical trajectory of the target foreign object to obtain the route selection for the target foreign object to move from one position to the next and the selection probability Among them, represents the selection of route i, represents that the target foreign object selects the route with a probability of; Classifying the foreign objects into periodic foreign objects and non-periodic foreign objects according to the appearance frequency and number of the foreign objects in the current scene, and obtaining the motion trajectory of the target foreign object after the current monitoring scene; wherein, the foreign objects include the target foreign object and other foreign objects; The probability p that the target foreign object selects line i in its movement trajectory after the current monitoring scenario i : Among them, w t is expressed as a probability of the set weight, w c is expressed as a probability p ci of the set weight, which represents the probability of other foreign objects choosing route i.

2. The method for detecting the movement trajectory of an object in a substation based on computer vision according to claim 1, wherein: Performing foreign object recognition on the collected scene video data to obtain foreign object pictures in the substation monitoring scene, specifically including: S11. Slicing the video data at a specified frame interval to form n pictures in time sequence; S12. Denote the nth picture and the (n - 1)th picture as F n and F n-1 respectively. Denote the gray values of the corresponding pixels of the two pictures as F n (x, y) and F n-1 (x, y). Subtract the gray values of the corresponding pixels of the two images and take the absolute value to obtain the difference image D n ; S13. Classify the gray value of any pixel in the differential image D n into one of the two values 0 or 255 to obtain a binary image R n '; S14. Perform connectivity analysis on the binarized image R n ′ to obtain the image R n that contains the complete moving foreign object; S15. Label m foreign objects in image R n and mark the area range where the foreign objects are located: Rang n (x n1 ,y n1 ,x n2 ,y n2 ); where m is an integer representing the total number of foreign objects recognized in the scene picture; in Rang n (x n1 ,y n1 ,x n2 ,y n2 ), n ∈ (1, 2,..., m), (x n1 , y n1 ) represents the upper left coordinate of the area where the nth foreign object is located, and (x n2 , y n2 ) represents the lower right coordinate of the area where the nth foreign object is located; S16. Segment the image regions of m foreign objects according to the range Rang of the area where the foreign objects are located n (x n1 , y n1 , x n2 , y n2 ) to obtain m pictures corresponding to the m foreign objects by performing image segmentation on the image regions of the m foreign objects respectively.

3. The method for detecting the movement trajectory of an object in a substation based on computer vision according to claim 1, characterized in that: Extracting the characteristic values of the foreign objects from the foreign object pictures, specifically including: Performing feature enhancement on the foreign object pictures to obtain enhanced foreign object pictures; Extracting texture features, edge features, and gray mean features from the enhanced foreign object pictures to obtain a feature point data set corresponding to the corresponding features, and using the feature point data set as the characteristic values of the foreign objects.

4. The method for detecting the movement trajectory of an object in a substation based on computer vision according to claim 3, wherein: Identifying the characteristic values of the foreign objects to obtain the type information of the foreign objects, specifically including: performing iterative operations on the feature point data set through the RANSAC algorithm, and finding the optimal parameters to minimize the cost function to obtain the type information corresponding to each foreign object; wherein, the type information includes at least one of a kite, a bird's nest, a hydrogen balloon, a branch, and a disposable plastic product.

5. The method for detecting the movement trajectory of an object in a substation based on computer vision according to claim 1, wherein: Judging whether the type information of the foreign object meets the preset storage condition, specifically including: Performing screening and verification on the type information of the foreign object through a preset screening strategy; When the type information of the foreign object does not exist in the database and there is no feature abnormality in the type information of the foreign object determined based on the preset screening strategy, it is determined that the type information of the foreign object meets the preset storage condition; wherein, the feature abnormality includes at least one of color or shape abnormality, abnormal recorded time format, and color or shape error exceeding a preset threshold.

6. The method for detecting the movement trajectory of an object in a substation based on computer vision according to claim 1, characterized in that: Based on the position information and time information in the motion data of the target foreign object, obtain the historical motion trajectory of the target foreign object, specifically including: Arrange the position information of the target foreign object in time series according to the position information and time information in the motion data of the target foreign object, and obtain the position information corresponding to the relevant time period; Based on the position information corresponding to the relevant time period, determine the historical motion trajectory of the target foreign object corresponding to the relevant time period.

7. The method for detecting the movement trajectory of objects in a substation based on computer vision according to claim 1, wherein: Determine the membership function corresponding to the foreign object flow density state in the monitoring scene, specifically including: Set the evaluation set V=(v1, v2, v3, v4) of the foreign object flow density state, where v1 represents not dense, v2 represents ordinary dense, v3 represents relatively dense, and v4 represents very dense; Let the membership degree of the \(i\)-th element in the factor set \(U\) to the \(j\)-th element in the evaluation set \(V\) be \(r\). ij , then the membership functions corresponding to the dense states of foreign object flow rates are as follows: Non-dense membership function r 11 (m): Membership function r of ordinary density 12 (m): Relatively dense membership function r 13 (m): Very dense membership function r 14 (m): Among them, m represents the total number of foreign objects in the scene.

8. A system for detecting the movement trajectory of objects in a substation based on computer vision, characterized in that: Including: Monitoring equipment, edge server, and data management server; The monitoring equipment is used to identify foreign objects from the collected scene video data and obtain foreign object pictures in the substation monitoring scene; The edge server is used to extract the feature values of the foreign objects from the foreign object pictures; identify the feature values of the foreign objects to obtain the type information of the foreign objects; The data management server is used to judge whether the type information of the foreign object meets the preset storage conditions. If so, enter the type information of the foreign object into the database; if not, do not process it; obtain the motion data matching the target foreign object to be measured from the database, and based on the position information and time information in the motion data of the target foreign object, obtain the historical motion trajectory of the target foreign object; based on the position information of the target foreign object in the current monitoring scene and the historical motion trajectory, determine the motion trajectory of the target foreign object after the current monitoring scene; Before extracting the feature values of the foreign objects from the foreign object pictures, it also includes: Determine the membership function corresponding to the foreign object flow density state in the monitoring scene according to the number of foreign object pictures obtained within the preset time period; among them, establish a comprehensive evaluation factor set U=(u1, u2, u3) according to the influencing factors of the foreign object flow, where u1 represents the foreign object density in the scene, u2 represents the current residence time, and u3 represents the foreign object motion time in the current scene; According to the membership function, determine the sending period of the foreign object pictures and send the foreign object pictures to the monitoring equipment at the sending period; Based on the position information of the target foreign object in the current monitoring scene and the historical motion trajectory, determine the motion trajectory of the target foreign object after the current monitoring scene, specifically including: Analyze the historical trajectory of the target foreign object to obtain the route selection of the target foreign object from a certain position to the next position and the selection probability wherein represents the selection of route i, represents the probability that the target foreign object selects route ; Classify the foreign objects into periodic foreign objects and non-periodic foreign objects according to the frequency and number of occurrences of the foreign objects in the current scene, and obtain the motion trajectory of the target foreign object after the current monitoring scene; among them, the foreign objects include the target foreign object and other foreign objects; The probability p of the target foreign object selecting line i in the movement trajectory after the current monitoring scenario i : Among them, w t is expressed as a probability of the set weight, w c is expressed as the probability p ci of the set weight, which represents the probability of other foreign objects choosing route i.

Images