Accurate positioning method for operation and maintenance personnel based on fusion of UWB and virtual and real scene similarity measurement
The integration of UWB with virtual and real scene similarity measurement optimizes positioning accuracy in substations by correcting UWB errors, addressing equipment obstruction and cost issues, ensuring safe and efficient operation.
Patent Information
- Application Number
- CN202111474924.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-03
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2041-12-03
AI Technical Summary
Existing UWB positioning technologies in complex substation environments suffer from reduced accuracy due to equipment obstruction, leading to difficulties in meeting the safety requirements for operation and maintenance personnel, and existing methods with high equipment requirements and costs.
A method integrating UWB positioning with virtual and real scene similarity measurement, utilizing a spatial error model and Monte Carlo simulations to optimize step lengths and image adjustments, enhancing positioning accuracy by correcting UWB results with virtual-real scene image similarity.
Improves positioning accuracy in substations by minimizing error and reducing equipment costs, providing precise location data for operation and maintenance personnel.
Smart Images
Figure CN114330502B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of space positioning, and in particular relates to a method for accurately positioning operation and maintenance personnel based on the fusion of UWB and virtual-real scene similarity measurement. Background Art
[0002] At present, the commonly used positioning technologies mainly include Wi-Fi, FRID, ultrasonic, infrared, ZigBee and Bluetooth. These positioning technologies have been widely used in airports, coal mines, hospitals, tunnels, construction sites and other occasions, and have achieved good application results. Ultra-WideBand (UWB) has a large bandwidth, strong time resolution and good anti-electromagnetic interference ability, which is suitable for the needs of substation inspection systems. In recent years, UWB has been widely used in power systems, but the substation equipment in the substation is dense and severely blocked, which greatly affects the accuracy of UWB positioning. Compared with WiFi and ZigBee, although the positioning accuracy of UWB has been greatly improved, the positioning accuracy of UWB positioning technology in the complex environment of substations cannot meet the actual positioning needs. In order to improve the positioning accuracy of substation operation and maintenance personnel, positioning methods combining UWB positioning with other methods have been continuously proposed, such as combining UWB positioning with power GIS, combining UWB with inertial navigation devices, etc., but these methods have high requirements on equipment and high operating costs. In addition, in the complex environment of substations, UWB positioning technology is affected by factors such as obstruction by substation equipment, resulting in reduced positioning accuracy and difficulty in meeting the actual requirements of substation operation and maintenance safety.
[0003] Due to the complexity of the real environment, the error model can only approximately reflect the relationship between the step size and the error, and the deviation between the model and the actual situation in the z-axis direction of the vertical height is relatively large; in the process of photo similarity matching, the candidate photos sampled from the virtual scene need to be stored, which takes up a large amount of storage space; if the virtual scene photo acquisition program and the similarity calculation program are not run on the same computer, it will also be affected by the data transmission rate.
[0004] The Chinese invention patent application (application publication number CN112367612A, application publication date 2021.02.12) discloses a UWB-based positioning method, UWB equipment and positioning system, and the Chinese invention patent (authorization announcement number CN111091170B, authorization announcement date 2020.8.25) discloses a site personnel management based on UWB positioning and millimeter wave imaging. The above-mentioned comparative documents all use a single UWB positioning, which is not suitable for the positioning of operation and maintenance personnel in complex substation environments. Summary of the invention
[0005] In view of the deficiencies of the above-mentioned technologies, the present invention provides a method for accurately positioning operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement, which can significantly improve the positioning accuracy.
[0006] In order to solve the above technical problems, the present invention provides a method for accurately locating operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement. The specific process is as follows:
[0007] Step 1) Establish spatial error model
[0008] The spatial error model is established through geometric relationship and probability error theory. The spatial error model is:
[0009]
[0010] And the probability p in the constraint condition is:
[0011] taken
[0012]
[0013] Among them, t0 is the time required for the computer to process an image, e0 is the error threshold, and p0 is the probability threshold;
[0014] -R x ~R x is the search range in the x direction, s x is the step length in the x direction; -R y ~R y is the search range in the y direction, s y is the step length in the y direction; -β yaw is the lower limit of the yaw angle (yaw) variation range, β yaw is the upper limit of the yaw angle (yaw) variation range, s yaw The step size for yaw angle change; -β pitch is the lower limit of the pitch angle variation range, β pitch is the upper limit of the pitch angle variation range, s pitch is the pitch angle change step size; t(s x ,s y ,s yaw ,s pitch ) is the setting time; C true is the real coordinate, is the true yaw angle, is the true pitch angle;
[0015] Step 2) Obtain the optimal x-direction step length s according to the spatial error model x and the y-direction step length s y , yaw angle step length syaw and pitch angle step length s pitch ;
[0016] Step 3) UWB preliminary positioning and virtual and real scene image acquisition and preprocessing;
[0017] First, the UWB positioning system is used to obtain the positioning information of the operation and maintenance personnel in the real substation scene, and the helmet-mounted camera worn by the operation and maintenance personnel is used to obtain the real substation scene image taken at the location; the location information of the operation and maintenance personnel located by UWB in the real substation scene is mapped to the virtual substation scene to obtain the location information of the operation and maintenance personnel in the virtual substation scene; then, the position mapped to the virtual substation scene is used as the reference point, and the position (x, y), yaw angle and pitch angle of the virtual camera are adjusted by the optimal x-direction step length, y-direction step length, yaw angle step length and pitch angle step length obtained in step 2), and a series of virtual substation scene images are taken, and the virtual substation scene images are processed; then, the real substation scene images are processed;
[0018] Step 4) Virtual and real scene similarity measurement and precise positioning
[0019] First, the virtual and real scene images obtained in step 3) are preliminarily screened, and then the virtual and real scene similarity is measured for the preliminarily screened results and the similarity score is calculated. The one with the highest score is the virtual substation scene image that is most similar to the real substation scene image; the shooting position of the virtual scene image that is determined to be most similar to the real scene image is obtained and mapped to the real substation scene, and the UWB positioning result of the operation and maintenance personnel in the substation is corrected; finally, the precise position of the operation and maintenance personnel in the substation is output.
[0020] Furthermore, in step 1), the specific calculation process of the error error is as follows:
[0021] The UWB positioning coordinates are mapped to the virtual scene to obtain the corresponding virtual positioning coordinates, and the virtual positioning coordinates are used as the relative coordinate origin of the virtual scene; if the virtual coordinates after the real coordinates are mapped to the virtual scene obey uniform distribution within the search range, the yaw angle and pitch angle of the real image also obey uniform distribution within the angle variation range; the Monte Carlo method is used to independently sample the real coordinates, real yaw angle and real pitch angle according to the uniform distribution to obtain m groups of samples, each group of samples includes the real coordinates C true , true yaw angle and the true pitch angle
[0022] C true ~U([-R x , R x ]×[-R y , R y ])
[0023]
[0024] After getting m groups of samples, record the error in these m groups of samples as The total number of groups is denoted as m′. According to the law of large numbers, when m→∞
[0025]
[0026] Then let f be the required probability p.
[0027] Furthermore, the specific process in step 2) is:
[0028] 2a) Let the distance from the virtual camera to the target image be L, and the half-angle of the virtual camera's yaw field of view be α yaw , when the yaw angle of the virtual camera changes α relative to the yaw centerline yaw When , the image deformation caused by the change of yaw angle is
[0029]
[0030] Similarly, when the virtual camera pitch field of view half angle is α pitch , the pitch angle of the virtual camera changes relative to the pitch centerline by α pitch When , the image deformation caused by the change of pitch angle is
[0031]
[0032] According to the real coordinates C true =(X true , Y true ), true yaw angle and the true pitch angle The candidate circle equation of the yaw angle is obtained as:
[0033]
[0034] In the formula, (X yaw , Y yaw ) is the coordinate of the center of the candidate circle of yaw angle, is the radius of the yaw angle candidate circle
[0035]
[0036]
[0037]
[0038] Where L is the distance from the virtual camera to the target mirror, α yawis the half-opening angle of the yaw field of view of the virtual camera;
[0039] Assume a two-dimensional search plane: the search range in the x direction is -R x ~R x , the search range in the y direction is -R y ~R y When the height of the inner camera is z=Z0, the candidate circle family of pitch angles can be expressed as
[0040]
[0041] In the formula, (Z pitch (Y), Y pitch (Y)) is the coordinate of the center of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle, is the radius of the candidate circle for the pitch angle
[0042]
[0043]
[0044]
[0045] In the formula, α pitch It is the half-open angle of the virtual camera's pitch field of view;
[0046] 2b) Use Monte Carlo method to obtain multiple sets of step lengths, each set of step lengths includes the x-direction step length s x , y direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch ; For each set of real coordinates C true and the actual yaw and pitch angles Determine the candidate yaw angle circle Cir yaw , the reference candidate point set on the yaw angle candidate circle is
[0047]
[0048] In the formula, (X yaw , Y yaw ) is the yaw angle - the coordinate of the center of the candidate circle, is the yaw angle-radius of the candidate circle, Yaw represents the yaw angle value set,
[0049] Then, at the position of each point in the reference candidate point set of the yaw angle candidate circle, the pitch angle candidate circle at that location is calculated, and the projection points of the coordinate points on the pitch angle candidate circle that satisfy the pitch angle step relationship on the xOy plane are taken and incorporated into the reference candidate point set; finally, the points within the search range [-Rx , R x ]×[-R y , R y ], the final reference candidate point set is
[0050]
[0051] In the formula, S pitch (X, Y) is the reference candidate point of the projection of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle
[0052]
[0053] Where x is the x-axis coordinate of the reference candidate point, and Pitch represents the pitch angle value set.
[0054]
[0055] 2c) After obtaining the final reference candidate point set, for each final reference candidate point, select the candidate coordinate set S cand (S cand ={(us x , vs y ): u = 0, ± 1, ..., ± [R x / s x ],v=0,±1,...,±[R y / s y ]}) are used as the envelope of the final reference candidate point; the envelope of all the final reference candidate points constitutes an envelope point set Enve, and the similarity distance DS between each candidate coordinate point in the envelope point set and the reference candidate point is calculated, and the candidate coordinate point with the smallest similarity distance is used as the final positioning coordinate of the algorithm;
[0056] For a candidate coordinate point C in the envelope point set Enve cand , the similarity distance DS consists of two parts: the distance from the candidate coordinate point to a reference candidate point C and the image deformation penalty term. Let θ yaw With θ pitch is the yaw angle and pitch angle corresponding to point C, and the image deformation penalty term is taken as So C cand The similarity distance is defined as
[0057]
[0058] In the formula, the minimum principal value argmin is the value of C∈S yaw-pitch Finally, the final positioning coordinate is the coordinate point with the smallest similarity distance in the envelope point set, that is,
[0059] C Alg =argmin DS(C cand )
[0060] So for a set of input C true and The positioning error of the algorithm is
[0061]
[0062] The error model is solved using the exterior point penalty function method and genetic algorithm to obtain the optimization result, that is, the optimal x-direction step length s x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch .
[0063] Furthermore, the specific process of step 3) is as follows:
[0064] First, the UWB positioning technology is used to obtain the location information of the operation and maintenance personnel, and the location coordinates are mapped to the virtual substation scene to obtain the corresponding coordinates in the virtual substation scene. Then, with the corresponding coordinates as the center, the x and y directions are -R x ~R x , -R y ~R y The range is the initial positioning range;
[0065] The acquisition and preprocessing of virtual and real scene images are as follows: within the initial positioning range corresponding to the virtual substation scene, the optimal x-direction step length s obtained in step 2) is used x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch The virtual camera coordinates (x, y) are moved to obtain a series of virtual images; and then the virtual substation scene images are processed.
[0066] Furthermore, the specific process of step 4) is as follows:
[0067] The initial screening operation in the measurement of similarity and precise positioning of virtual and real scenes is: calculate the similarity of the processed series of virtual substation scene images A with the processed real substation scene image T. First, perform the initial screening: take the strip sub-area Lmargin_A of 1 / 20 on the left side of A and the strip sub-area Rmargin_A of 1 / 20 on the right side, and take the strip sub-area Lmargin_T of 1 / 20 on the left side of T and the strip sub-area Rmargin_T of 1 / 20 on the right side; calculate the proportion Lratio_A of the pixel value difference between Lmargin_A and Lmargin_T that is greater than the given threshold x, and the proportion Rratio_A of the pixel value difference between Rmargin_A and Rmargin_T that is greater than the given threshold x; then sort the virtual substation scene images from small to large according to ratio_A, and select the top 5% of virtual substation scene images as the initial screening results;
[0068] The similarity measurement and precise positioning operation of virtual and real scenes are as follows: Perform Canny edge detection on images A and T in the initial screening results to obtain edge images and To measure the similarity between two images, this paper uses the Frobenius-norm (F-norm) for similarity measurement, and the difference between images A and T is defined as
[0069]
[0070] Where λ is the edge similarity coefficient; the similarity score between images A and B is defined as
[0071]
[0072] The image similarity score is calculated by S(A, T), and the coordinates of the virtual image with the highest similarity score are selected and mapped to the real substation scene as the precise positioning coordinates.
[0073] Compared with the prior art, the present invention has the following advantages:
[0074] 1) In the complex environment of substations, traditional UWB positioning technology is affected by factors such as obstruction by substation equipment, resulting in reduced positioning accuracy and difficulty in meeting the actual requirements of substation operation and maintenance safety; the present invention uses a positioning method that integrates UWB and virtual and real scene similarity measurement to effectively improve the positioning accuracy of substation operation and maintenance personnel and meet actual operation and maintenance needs.
[0075] 2) Positioning methods such as combining UWB positioning with power GIS and combining UWB with inertial navigation devices have high requirements on equipment and high operating costs. The present invention uses similarity measurement of virtual and real scene images to assist UWB positioning, which has low requirements on equipment and can be achieved on an ordinary computer, thereby reducing operating costs.
[0076] 3) In the present invention, a spatial error model is established, and the positioning method of the fusion of UWB and virtual-real scene similarity measurement is theoretically analyzed using geometric relationship and probability error theory. The positioning time is minimized within the allowable error. The method of fusion of UWB and virtual-real scene similarity measurement is used to overcome the influence of factors such as occlusion on the UWB positioning accuracy, significantly improve the positioning accuracy, and provide technical support for intelligent operation and maintenance and safe production of substations. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] Figure 1 is a schematic diagram of yaw angle-candidate circle;
[0078] Figure 2 is a schematic diagram of the relationship between the yaw angle and the candidate circle and the pitch angle and the candidate circle;
[0079] Figure 3 It is a schematic diagram of a certain yaw angle-candidate circle. DETAILED DESCRIPTION
[0080] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0081] The precise positioning method of operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement is as follows:
[0082] Step 1) Establish spatial error model
[0083] The spatial error model is established through geometric relationship and probability error theory. The spatial error model is:
[0084]
[0085] And the probability p in the constraint condition is:
[0086] taken
[0087]
[0088] Among them, t0 is the time required for the computer to process an image, e0 is the error threshold, and p0 is the probability threshold;
[0089] -R x ~R x is the search range in the x direction, s x is the step length in the x direction; -R y ~R y is the search range in the y direction, s y is the step length in the y direction; -β yaw is the lower limit of the yaw angle (yaw) variation range, β yawis the upper limit of the yaw angle (yaw) variation range, s yaw The step size for yaw angle change; -β pitch is the lower limit of the pitch angle variation range, β pitch is the upper limit of the pitch angle variation range, s pitch is the pitch angle change step size; t(s x ,s y ,s yaw ,s pitch ) is the setting time; C true is the real coordinate, is the true yaw angle, is the actual pitch angle.
[0090] The specific calculation process of error error is as follows:
[0091] After mapping the UWB positioning coordinates to the virtual scene, the corresponding virtual positioning coordinates are obtained, and the virtual positioning coordinates are used as the relative coordinate origin of the virtual scene; if the virtual coordinates after the real coordinates are mapped to the virtual scene are within the search range (the search range in the x direction is -R x ~R x , the search range in the y direction is -R y ~R y ) is uniformly distributed within the range of yaw angle and pitch angle of the real image, then the yaw angle and pitch angle of the real image are within the range of yaw angle (i.e. the yaw angle range is -β yaw ~β yaw , the pitch angle changes within the range of -β pitch ~β pitch ) also obeys uniform distribution; Monte-Carol method is used to independently sample the true coordinates, true yaw angle and true pitch angle according to uniform distribution, and m groups of samples are obtained, each group of samples includes the true coordinates C true , true yaw angle and the true pitch angle
[0092] C true ~U([-R x , R x ]×[-R y , R y ])(that is, the real coordinates are evenly distributed within the search range) (That is, the true yaw angle and true pitch angle are evenly distributed within the angle variation range)
[0093] After getting m groups of samples, record the error in these m groups of samples as The total number of groups is denoted as m′. According to the law of large numbers, when n→∞
[0094]
[0095] So let f be the required probability p;
[0096] Step 2) Obtain the optimal x-direction step length s according to the spatial error model x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch
[0097] 2a) Let the distance from the virtual camera to the target image be L, and the half-angle of the virtual camera's yaw field of view be α yaw , when the yaw angle of the virtual camera changes by α relative to the yaw centerline (yaw angle is zero) yaw When , the image deformation caused by the change of yaw angle is
[0098]
[0099] Similarly, when the virtual camera pitch field of view half angle is α pitch , the pitch angle of the virtual camera changes by α relative to the pitch centerline (pitch angle is zero) pitch When , the image deformation caused by the change of pitch angle is
[0100]
[0101] Calculate the candidate circles of yaw angle and pitch angle, such as Figure 1 As shown in the figure, when the virtual camera moves on the candidate yaw angle circle with the x-axis projection of the real image as the chord and the circle where the real coordinates are located, the virtual image captured by the virtual camera has the same information in the x-axis direction, that is, the position and size of the virtual image in the virtual scene are the same, and only differ by a deformation factor. According to the real coordinate C true =(X true , Y true ), true yaw angle and the true pitch angle The candidate circle equation of the yaw angle is obtained as:
[0102]
[0103] In the formula, (X yaw , Y yaw ) is the coordinate of the center of the candidate circle of yaw angle, is the radius of the yaw angle candidate circle
[0104]
[0105]
[0106]
[0107] Where L is the distance from the virtual camera to the target mirror, α yaw is the half-opening angle of the yaw field of view of the virtual camera;
[0108] The virtual camera moves in the height direction, i.e., the z direction. After the search space of the virtual camera is expanded into a three-dimensional space, there will be a pitch angle candidate circle at the position of each point on the yaw angle candidate circle. These pitch angle candidate circles form a pitch angle candidate circle family. The pitch angle candidate circle family is orthogonal to the yaw angle candidate circle, such as Figure 2 , suppose the two-dimensional search plane (i.e. the search range in the x direction is -R x ~R x , the search range in the y direction is -R y ~R y ) When the camera height z = Z0, the candidate circle family of pitch angles can be expressed as
[0109]
[0110] In the formula, (Z pitch (Y), Y pitch (Y)) is the coordinate of the center of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle, is the radius of the candidate circle for the pitch angle
[0111]
[0112]
[0113]
[0114] In the formula, α pitch It is the half-open angle of the virtual camera's pitch field of view.
[0115] 2b) Use Monte Carlo method to obtain multiple sets of step lengths, each set of step lengths includes the x-direction step length s x , y direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch ; For each set of real coordinates C true and the actual yaw and pitch angles Determine the candidate yaw angle circle Cir yaw ,like Figure 3 , and then select the coordinate points on the yaw angle candidate circle whose yaw angles satisfy the yaw angle step length relationship as reference candidate points, and the reference candidate point set on the yaw angle candidate circle is
[0116]
[0117] In the formula, (X yaw , Y yaw ) is the yaw angle - the coordinate of the center of the candidate circle, is the yaw angle-radius of the candidate circle, Yaw represents the yaw angle value set,
[0118] Then, at the position of each point in the reference candidate point set of the yaw angle candidate circle, the pitch angle candidate circle at that location is calculated, and the projection points of the coordinate points on the pitch angle candidate circle that satisfy the pitch angle step relationship on the xOy plane are taken and incorporated into the reference candidate point set; finally, the points within the search range [-R x , R x ]×[-R y , R y ], the final reference candidate point set is
[0119]
[0120] In the formula, S pitch (X, Y) is the reference candidate point of the projection of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle
[0121]
[0122] Where x is the x-axis coordinate of the reference candidate point, and Pitch represents the pitch angle value set.
[0123]
[0124] 2c) After obtaining the final reference candidate point set, for each final reference candidate point, select the candidate coordinate set S cand (S cand ={(us x , vs y ): u = 0, + 1, ..., ± [R x / s x ],v=0,±1,...,±[R y / s y ]}) The four coordinate points closest to the final reference candidate point are used as the envelope of the final reference candidate point. The envelope of all final reference candidate points constitutes an envelope point set Enve, and the similarity distance DS between each candidate coordinate point in the envelope point set and the reference candidate point is calculated, and the candidate coordinate point with the smallest similarity distance is used as the final positioning coordinate of the algorithm.
[0125] For a candidate coordinate point C in the envelope point set Enve cand, the similarity distance DS should consist of two parts: the distance from the candidate coordinate point to a reference candidate point C and the image deformation penalty term. Let θ yaw With θ pitch is the yaw angle and pitch angle corresponding to point C, and the image deformation penalty term is taken as So C cand The similarity distance is defined as
[0126]
[0127] In the formula, the minimum principal value argmin is the value of C∈S yaw-pitch Finally, the final positioning coordinate is the coordinate point with the smallest similarity distance in the envelope point set, that is,
[0128] C Alg =argmin DS(C cand )
[0129] So for a set of input C true and The positioning error of the algorithm is
[0130]
[0131] The error model is solved using the exterior point penalty function method and genetic algorithm to obtain the optimization result, that is, the optimal x-direction step length s x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch .
[0132] Step 3) UWB preliminary positioning and virtual and real scene image acquisition and preprocessing
[0133] First, the UWB positioning system is used to obtain the positioning information of the operation and maintenance personnel in the real substation scene, and the helmet-mounted camera worn by the operation and maintenance personnel is used to obtain the real substation scene image taken at this location; the location information of the operation and maintenance personnel UWB positioning in the real substation scene is mapped to the virtual substation scene to obtain the location information of the operation and maintenance personnel in the virtual substation scene; then, taking the position mapped to the virtual substation scene as the reference point, the position (x, y), yaw angle and pitch angle of the virtual camera are adjusted by the optimal x-direction step length, y-direction step length, yaw angle step length and pitch angle step length obtained in step 2), a series of virtual substation scene images are taken, and the virtual substation scene images are processed (color space conversion, grayscale transformation normalization, edge detection); then the real substation scene images are processed (color space conversion, grayscale transformation normalization, edge detection, k-means clustering, optical flow analysis, feature point matching).
[0134] 3a) The operation of UWB preliminary positioning technology is as follows: first, the location information of the operation and maintenance personnel is obtained by using UWB positioning technology, and the location coordinates are mapped to the virtual substation scene to obtain the corresponding coordinates in the virtual substation scene, and then the corresponding coordinates are used as the center to locate the position of the operation and maintenance personnel in the x and y directions. x ~R x , -R y ~R y The range is the initial positioning range;
[0135] The acquisition and preprocessing of virtual and real scene images are as follows: within the initial positioning range corresponding to the virtual substation scene, the optimal x-direction step length s obtained in step 2) is used x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch Move the virtual camera coordinates (x, y) to obtain a series of virtual images. Then process the real substation scene images (color space conversion, grayscale transformation normalization, edge detection, k-means clustering, optical flow analysis, feature point matching); process the virtual substation scene images (color space conversion, grayscale transformation normalization, edge detection).
[0136] Step 4) Virtual and real scene similarity measurement and precise positioning
[0137] First, the virtual and real scene images obtained in step 3) are preliminarily screened, and then the virtual and real scene similarity is measured for the preliminarily screened results and the similarity score is calculated. The one with the highest score is the virtual substation scene image that is most similar to the real substation scene image; the shooting position of the virtual scene image that is determined to be most similar to the real scene image is obtained and mapped to the real substation scene, and the UWB positioning result of the operation and maintenance personnel in the substation is corrected; finally, the precise position of the operation and maintenance personnel in the substation is output.
[0138] The initial screening operation in the virtual-real scene similarity measurement and precise positioning is: the similarity of the processed series of virtual substation scene images A and the processed real substation scene images T is calculated respectively. First, the initial screening is performed: take the strip sub-area Lmargin_A of 1 / 20 on the left side of A and the strip sub-area Rmargin_A of 1 / 20 on the right side, take the strip sub-area Lmargin_T of 1 / 20 on the left side of T and the strip sub-area Rmargin_T of 1 / 20 on the right side; calculate the proportion Lratio_A of the pixel value difference between Lmargin_A and Lmargin_T greater than the given threshold x, and the proportion Rratio_A of the pixel value difference between Rmargin_A and Rmargin_T greater than the given threshold x; then sort the virtual substation scene images from small to large according to ratio_A (ratio_A=Lratio_A+Rratio_A), and select the top 5% of virtual substation scene images as the initial screening results.
[0139] The similarity measurement and precise positioning operation of virtual and real scenes are as follows: Perform Canny edge detection on images A and T in the initial screening results to obtain edge images and To measure the similarity between two images, this paper uses the Frobenius-norm (F-norm) for similarity measurement, and the difference between images A and T is defined as
[0140] (F is the symbol for norm)
[0141] Where λ is the edge similarity coefficient; the similarity score between images A and B is defined as
[0142]
[0143] The image similarity score is calculated by S(A,T), and the coordinates of the virtual image with the highest similarity score are selected and mapped to the real substation scene as the precise positioning coordinates.
[0144] The contents not described in detail in this specification belong to the prior art known to the professional and technical personnel in this field. It should be understood that those of ordinary skill in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A precise positioning method for operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement. The specific process is as follows: Step 1) Establish spatial error model The spatial error model is established through geometric relationship and probability error theory. The spatial error model is: And the probability p in the constraint condition is: Taken in, t0 is the time required for the computer to process an image, e0 is the error threshold, and p0 is the probability threshold; -R x ~R x is the search range in the x direction, s x is the step length in the x direction; -R y ~R y is the search range in the y direction, s y is the step length in the y direction; -β yaw is the lower limit of the yaw angle (yaw) variation range, β yaw is the upper limit of the yaw angle (yaw) variation range, s yaw The step size for yaw angle change; -β pitch is the lower limit of the pitch angle variation range, β pitch is the upper limit of the pitch angle variation range, s pitch is the pitch angle change step size; t(s x ,s y ,s yaw ,s pitch ) is the setting time; C true is the real coordinate, is the true yaw angle, is the true pitch angle; Step 2) Obtain the optimal x-direction step length s according to the spatial error model x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch ; Step 3) UWB preliminary positioning and virtual and real scene image acquisition and preprocessing; First, the UWB positioning system is used to obtain the positioning information of the operation and maintenance personnel in the real substation scene, and the helmet-mounted camera worn by the operation and maintenance personnel is used to obtain the real substation scene image taken at the location; the location information of the operation and maintenance personnel UWB positioning in the real substation scene is mapped to the virtual substation scene to obtain the location information of the operation and maintenance personnel in the virtual substation scene; then, the position mapped to the virtual substation scene is used as the reference point, and the position (x, y), yaw angle and pitch angle of the virtual camera are adjusted by the optimal x-direction step length, y-direction step length, yaw angle step length and pitch angle step length obtained in step 2), and a series of virtual substation scene images are taken, and the virtual substation scene images are processed; then the real substation scene images are processed; Step 4) Virtual and real scene similarity measurement and precise positioning First, the virtual and real scene images obtained in step 3) are preliminarily screened, and then the virtual and real scene similarity is measured for the preliminarily screened results and the similarity score is calculated. The one with the highest score is the virtual substation scene image that is most similar to the real substation scene image; the shooting position of the virtual scene image that is determined to be most similar to the real scene image is obtained and mapped to the real substation scene, and the UWB positioning result of the operation and maintenance personnel in the substation is corrected; finally, the precise position of the operation and maintenance personnel in the substation is output.
2. The method for accurately locating operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement according to claim 1 is characterized by: In step 1), the specific calculation process of the error error is as follows: The UWB positioning coordinates are mapped to the virtual scene to obtain the corresponding virtual positioning coordinates, and the virtual positioning coordinates are used as the relative coordinate origin of the virtual scene; if the virtual coordinates after the real coordinates are mapped to the virtual scene obey uniform distribution within the search range, the yaw angle and pitch angle of the real image also obey uniform distribution within the angle variation range; the Monte Carlo method is used to independently sample the real coordinates, real yaw angle and real pitch angle according to the uniform distribution to obtain m groups of samples, each group of samples includes the real coordinates C true , true yaw angle and the true pitch angle C true ~U[-R x ,R x ]×[-R y ,R y ]) After getting m groups of samples, record the error in these m groups of samples as The total number of groups is denoted as m′. According to the law of large numbers, when m→∞ Then let f be the required probability p.
3. The method for accurately locating operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement according to claim 1 is characterized by: The specific process in step 2) is: 2a) Let the distance from the virtual camera to the target image be L, and the half-angle of the virtual camera's yaw field of view be α yaw , when the yaw angle of the virtual camera changes θ relative to the yaw centerline yaw When , the image deformation caused by the change of yaw angle is Similarly, when the virtual camera pitch field of view half angle is α pitch , the pitch angle of the virtual camera changes relative to the pitch centerline by θ pitch When , the image deformation caused by the change of pitch angle is According to the real coordinates C true =(X true , Y true ), true yaw angle and the true pitch angle The candidate circle equation of the yaw angle is obtained as: In the formula, (X yaw , Y yaw ) is the coordinate of the center of the candidate circle of yaw angle, is the radius of the candidate circle for the yaw angle Where L is the distance from the virtual camera to the target mirror, α yaw is the half-opening angle of the yaw field of view of the virtual camera; Assume a two-dimensional search plane: the search range in the x direction is -R x ~R x , the search range in the y direction is -R y ~R y When the height of the inner camera is z=Z0, the candidate circle family of pitch angles can be expressed as In the formula, (Z pitch (Y), Y pitch (Y)) is the coordinate of the center of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle, is the radius of the candidate circle for the pitch angle In the formula, a pitch It is the half-open angle of the virtual camera's pitch field of view; 2b) Use Monte Carlo method to obtain multiple sets of step lengths, each set of step lengths includes the x-direction step length s x , y direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch ; For each set of real coordinates C true and the actual yaw and pitch angles Determine the candidate yaw angle circle Cir yaw , the reference candidate point set on the yaw angle candidate circle is In the formula, (X yaw , Y yaw ) is the yaw angle - the coordinate of the center of the candidate circle, is the yaw angle-radius of the candidate circle, Yaw represents the yaw angle value set, Then, at the position of each point in the reference candidate point set of the yaw angle candidate circle, the pitch angle candidate circle at that location is calculated, and the projection points of the coordinate points on the pitch angle candidate circle that satisfy the pitch angle step relationship on the xOy plane are taken and incorporated into the reference candidate point set; finally, the points within the search range [-R x , R x ]×[-R y , R y ], the final reference candidate point set is In the formula, S pitch (X, Y) is the reference candidate point of the projection of the pitch angle candidate circle at (X, Y) on the yaw angle candidate circle Where x is the x-axis coordinate of the reference candidate point, and Pitch represents the pitch angle value set. 2c) After obtaining the final reference candidate point set, for each final reference candidate point, select the candidate coordinate set S cand (S cand ={(us x ,vs y ):u=0,±1,...,±[R x / s x ],v=0,±1,...,±[R y / s y ]}) the four coordinate points closest to the final reference candidate point are used as the envelope of the final reference candidate point; The envelope of all final reference candidate points constitutes the envelope point set Enev. The similarity distance DS between each candidate coordinate point in the envelope point set and the reference candidate point is calculated, and the candidate coordinate point with the smallest similarity distance is used as the final positioning coordinate of the algorithm. For a candidate coordinate point C in the envelope point set Enve eand , the similarity distance DS consists of two parts: the distance from the candidate coordinate point to a reference candidate point C and the image deformation penalty term; let θ yaw With θ pitch is the yaw angle and pitch angle corresponding to point C, and the image deformation penalty term is taken as So C cand The similarity distance is defined as In the formula, the minimum principal value argmin is the value of C∈S yaw-pitch Finally, the final positioning coordinate is the coordinate point with the smallest similarity distance in the envelope point set, that is, C Alg =argmin DS(C cand ) So for a set of input C true and The positioning error of the algorithm is The error model is solved using the exterior point penalty function method and genetic algorithm to obtain the optimization result, that is, the optimal x-direction step length s x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch .
4. The method for accurately locating operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement according to claim 1 is characterized by: The specific process of step 3) is as follows: First, the UWB positioning technology is used to obtain the location information of the operation and maintenance personnel, and the location coordinates are mapped to the virtual substation scene to obtain the corresponding coordinates in the virtual substation scene. Then, with the corresponding coordinates as the center, the x and y directions are -R x ~R x , -R y ~R y The range is the initial positioning range; The acquisition and preprocessing of virtual and real scene images are as follows: within the initial positioning range corresponding to the virtual substation scene, the optimal x-direction step length s obtained in step 2) is used x and the y-direction step length s y , yaw angle step length s yaw and pitch angle step length s pitch The virtual camera coordinates (x, y) are moved to obtain a series of virtual images; and then the virtual substation scene images are processed.
5. The method for accurately locating operation and maintenance personnel based on the fusion of UWB and virtual and real scene similarity measurement according to claim 1 is characterized in that: The specific process of step 4) is as follows: The initial screening operation in the similarity measurement and precise positioning of virtual and real scenes is as follows: the similarity of the processed series of virtual substation scene images A and the processed real substation scene image T is calculated respectively; firstly, the initial screening is performed: the strip sub-area Lmargin_A of 1 / 20 on the left side of A and the strip sub-area Rmargin_A of 1 / 20 on the right side, and the strip sub-area Lmargin_T of 1 / 20 on the left side of T and the strip sub-area Rmargin_T of 1 / 20 on the right side; the proportion Lratio_A of the difference in pixel values between Lmargin_A and Lmargin_T greater than the given threshold x, and the proportion Rratio_A of the difference in pixel values between Rmargin_A and Rmargin_T greater than the given threshold x are calculated; then the virtual substation scene images are sorted from small to large according to ratio_A, and the top 5% of the virtual substation scene images are selected as the initial screening results; The similarity measurement and precise positioning operation of virtual and real scenes are as follows: Perform Canny edge detection on images A and T in the initial screening results to obtain edge images and To measure the similarity between two images, this paper uses the Frobenius-norm (F-norm) for similarity measurement, and the difference between images A and T is defined as Where λ is the edge similarity coefficient; the similarity score between images A and B is defined as The image similarity score is calculated by S(A,T), and the coordinates of the virtual image with the highest similarity score are selected and mapped to the real substation scene as the precise positioning coordinates.