Power distribution board display method and device based on computer vision

Abstract

The invention provides an electric power distribution board display method and device based on computer vision. The method comprises the following steps: acquiring a multi-angle entity image of an electric power distribution board by using camera equipment; extracting matched feature points in the multi-angle entity image, and mapping the matched feature points into a three-dimensional sparse point cloud according to the space coordinates of the camera device; performing point cloud densification on the three-dimensional sparse point cloud to obtain a dense point cloud; constructing a distribution board static model of the power distribution board according to the dense point cloud; and displaying the operation information, obtained in real time, of the power distribution board in the static model of the power distribution board. Through the method, the operation data of the power distribution board are mapped to the static model of the power distribution board in real time, the remote intelligent monitoring and management level is improved, the power grid operation decision is optimized, and asset digitalization of the power distribution board is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twins, and in particular to a method and device for displaying a power distribution panel based on computer vision. Background Art

[0002] In the process of new power system construction and digital transformation, power communication equipment such as power distribution panels play an irreplaceable role as the basic facilities of the power industry. In the operation and maintenance business scenarios of communication equipment, the demand for digitalization of communication equipment assets is getting higher and higher. There are a lot of mechanical, repetitive, difficult to control operational risks, and time-consuming operations in the front-line equipment of communication production. At present, it is urgent to carry out research on the digital application of power communication assets to improve the efficiency of overall management. Summary of the invention

[0003] The present invention provides a method and device for displaying a power distribution panel based on computer vision, which is used to map the operating data of the power distribution panel to a static model of the distribution panel in real time, improve the level of remote intelligent monitoring and management, optimize power grid operation decisions, and realize the asset digitization of the power distribution panel.

[0004] The present invention provides a method for displaying a power distribution panel based on computer vision, comprising: Use camera equipment to obtain multi-angle physical images of the power distribution panel; Extracting matching feature points in the multi-angle entity image, and mapping the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device; Densifying the three-dimensional sparse point cloud to obtain a dense point cloud; constructing a static model of the power distribution panel of the power distribution panel according to the dense point cloud; The operation information of the power distribution panel obtained in real time is displayed in the static model of the power distribution panel.

[0005] According to a computer vision-based power distribution panel display method provided by the present invention, the matching feature points in the multi-angle entity image are extracted, and the matching feature points are mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device, including: Identify the distribution panel area in the multi-angle entity image; Segment the distribution panel area according to the GrabCut algorithm to obtain a foreground image of the power distribution panel; Extracting dense feature points in the foreground image, and performing feature point pair matching on the dense feature points to obtain matching feature points; Eliminating mismatched feature points in the matching feature point pairs; According to the spatial coordinates of the camera device, the matching feature points excluding the mismatched feature points are mapped into a three-dimensional sparse point cloud.

[0006] According to a computer vision-based power distribution panel display method provided by the present invention, the three-dimensional sparse point cloud is densified to obtain a dense point cloud, including: Obtain depth maps of multi-angle entity images at different angles; According to the image domain information obtained in the depth map, the three-dimensional sparse point cloud is subjected to patch expansion and patch filtering to obtain the dense point cloud.

[0007] According to a computer vision-based power distribution panel display method provided by the present invention, the method of constructing a static model of the power distribution panel according to the dense point cloud comprises: The reprojection error of the dense point cloud is constrained according to the light speed adjustment method, and the dense point cloud is reconstructed according to the Poisson surface reconstruction algorithm to obtain the static model of the distribution panel.

[0008] According to a computer vision-based power distribution panel display method provided by the present invention, the method of extracting dense feature points in a foreground image and performing feature point pair matching on the dense feature points to obtain a plurality of matching feature points includes: Using Harris corner detection algorithm to extract dense feature points in the foreground image; The KLT algorithm is used to track and match dense feature points to obtain multiple matching feature points.

[0009] According to a computer vision-based power distribution panel display method provided by the present invention, before extracting dense feature points in the foreground image, the method further includes: The image contrast of the foreground image is enhanced using an emphasize operator.

[0010] According to a computer vision-based power distribution panel display method provided by the present invention, the step of removing mismatched feature points from the matching feature points includes: Selecting a preset number of sample feature points from the matching feature points, wherein the preset number of sample feature points are not collinear; Constructing a matrix model according to the preset number of sample feature points; Calculating the projection error between each of the matching feature points and the matrix model; If the projection error is greater than or equal to a preset error threshold, the matching feature point is removed as a mismatched feature point.

[0011] A computer vision-based power distribution panel display method provided by the present invention also includes: The SURF algorithm is used to extract sparse feature points in the foreground image of the power distribution panel to obtain the spatial coordinates of the camera device.

[0012] The present invention also provides a computer vision-based power distribution panel display device, comprising: An image acquisition unit, used to acquire multi-angle physical images of the power distribution panel; A feature point matching unit, used to extract matching feature points in the multi-angle entity image, and map the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device; A feature point processing unit, used for densifying the three-dimensional sparse point cloud to obtain a dense point cloud; A model building unit, used for building a static model of the power distribution panel of the power distribution panel according to the dense point cloud; The information display unit is used to display the operation information of the power distribution panel obtained in real time into the static model of the power distribution panel.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the computer vision-based power distribution panel display method as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described computer vision-based power distribution panel display methods.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-described computer vision-based power distribution panel display methods.

[0016] The computer vision-based power distribution panel display method and device provided by the present invention constructs a static model of the distribution panel through computer vision technology, and integrates the real-time operation information of the power distribution panel into the static model of the distribution panel, thereby improving the intuitiveness and efficiency of operation and maintenance management. It realizes the acquisition, processing and display of multi-angle entity images of the power distribution panel, and improves the digital level of the power distribution panel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of a method for displaying a power distribution panel based on computer vision provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the principle of operating information identification of a power distribution panel provided by the present invention.

[0020] Figure 3 This is the second flow chart of a method for displaying a power distribution panel based on computer vision provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the principle of obtaining a three-dimensional sparse point cloud based on the triangulation principle provided by the present invention.

[0022] Figure 5 This is a flowchart of a method for displaying a power distribution panel based on computer vision provided by the present invention.

[0023] Figure 6 This is the fourth flow chart of a method for displaying a power distribution panel based on computer vision provided by the present invention.

[0024] Figure 7 It is a schematic diagram of the principle of a static model of a distribution panel provided by the present invention.

[0025] Figure 8 It is a result schematic diagram of a power distribution panel display device based on computer vision provided by the present invention.

[0026] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0029] Combine the following Figure 1-Figure 9 The specific contents of the present invention are described.

[0030] Figure 1 One of the flow charts of a method for displaying a power distribution panel based on computer vision provided in an embodiment of the present invention includes steps S101-S105.

[0031] Step S101, using a camera device to obtain multi-angle physical images of the power distribution panel.

[0032] In an embodiment of the present invention, the imaging device includes but is not limited to a camera, and the camera is used to acquire multi-angle physical entity images of the power distribution panel to be reconstructed to form an initial image data set.

[0033] Step S102 : extract matching feature points from the multi-angle entity image, and map the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0034] Step S103, densifying the three-dimensional sparse point cloud to obtain a dense point cloud.

[0035] Step S104, constructing a static model of the power distribution panel according to the dense point cloud.

[0036] Step S105, displaying the real-time operation information of the power distribution panel in the static model of the distribution panel.

[0037] The following is a description of a process for acquiring the operating information of a power distribution panel in an embodiment of the present invention.

[0038] In an embodiment of the present invention, a camera is used to capture a color image of the front of a power distribution panel in real time in a timed sequence. Objects such as switches, terminals, text, status lights, etc. are identified and interest frames are located and extracted from the real-time image based on the Faster R-CNN deep learning network, and the anchor frames of each object are analyzed to obtain the operation information of the distribution panel.

[0039] Exemplarily, a certain number of distribution panel images are obtained, and the labelimg toolbox is used to calibrate the objects representing the operating status, such as switches, terminals, text, status lights, etc., to produce an image training set; the above image training set is input into the Faster R-CNN network for training; the image set to be identified is input, and the Faster R-CNN network is used to identify the operating status of each object, such as switches, terminals, text, status lights, etc.

[0040] like Figure 2 As shown, for the text area image selected by the interest anchor frame, the OCR technology is used to extract and recognize the text in it, and the working status of the distribution panel is accurately obtained; specifically, it includes: using a convolutional recurrent neural network to extract image text features and recognize text content; using a connected temporal classifier to merge repeated characters, remove empty characters, and output the recognized text results.

[0041] Finally, the text results are displayed in the static model of the power distribution panel as the operation information of the power distribution panel.

[0042] Figure 3 The second flowchart of a method for displaying a power distribution panel based on computer vision provided in an embodiment of the present invention.

[0043] In a possible implementation, in step S102, matching feature points in the multi-angle entity image are extracted, and the matching feature points are mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device, such as Figure 3 As shown, steps S301-S305 are included.

[0044] Step S301, identifying the distribution panel area in the multi-angle entity image.

[0045] In an embodiment of the present invention, the Faster R-CNN network is used to identify the distribution panel in the initial image and extract the anchor frame area image, that is, the distribution panel area. Specifically, it includes: using the labelimg toolbox to calibrate the anchor frame of the area occupied by the multi-angle entity image, and making an image training set of the power distribution panel; inputting the above image training set into the Faster R-CNN network for training; inputting the image set to be identified, and using the Faster R-CNN network to frame the distribution panel area in the multi-angle entity image.

[0046] Step S302, segmenting the distribution panel area according to the GrabCut algorithm to obtain a foreground image of the power distribution panel.

[0047] Specifically, the GrabCut algorithm is used to perform semantic segmentation on the image in the anchor frame to remove the background and obtain the foreground image of the distribution panel.

[0048] Exemplarily, the image of the distribution panel area obtained in step S301 is used as input to define the foreground and background areas; Gaussian mixture modeling (GMM) is performed on the pixels in the image of the distribution panel area, and the foreground and background pixels are clustered into K categories using the K-means algorithm, which are used as K initialization models in the GMM; a minimization energy formula as shown below is constructed, and the image segmentation is completed by continuously iterative optimization based on the minimum cut or maximum flow algorithm so that the energy approaches the minimum value.

[0049]

[0050] Among them,  is the transparency coefficient, the background is set to 0 and the foreground is set to 1; k∈{1,2,···,K} is the number of GMM models; θ=(μ,δ2) is the mean and covariance of GMM; z represents the pixel; D represents the matching degree between the transparency coefficient  and the pixel z, which is minimized when the classification is correct; V is the boundary term, which reflects the similarity between adjacent pixels and is minimized at the image boundary.

[0051] Step S303: extract dense feature points in the foreground image, and perform feature point pair matching on the dense feature points to obtain matching feature point pairs.

[0052] Step S304: remove mismatched feature points from the matching feature point pairs.

[0053] Step S305 : mapping the matching feature points excluding the mismatched feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0054] In the embodiment of the present application, the Harris corner detection algorithm is used to extract dense feature points and combined with the KLT tracking algorithm to complete the matching of feature point pairs, and then the Ransac random consistent sampling algorithm is used to eliminate mismatched feature points. Finally, based on the triangulation principle, Figure 4As shown, the three-dimensional spatial coordinates of the feature points can be calculated using the spatial coordinates of the camera and the pixel coordinates of the matching feature points, and a three-dimensional sparse point cloud can be mapped. For details, please refer to the following embodiment.

[0055] In a possible implementation, dense feature points are extracted from the foreground image, and feature point pairs are matched on the dense feature points to obtain matching feature point pairs, including step S303A and step S303B.

[0056] Step S303A, using Harris corner detection algorithm to extract dense feature points in the foreground image.

[0057] Specifically, the window function ω(x, y) with the center as the coordinate (x, y) is used as a window to slide slightly in both the x and y directions of the foreground image, and the change in the pixel gray value E(u, v) inside the window is calculated, and the two-dimensional Taylor formula is expanded; the corner point response function in the window is calculated: ; Where H is the corner response function; λ1 and λ2 are the eigenvalues ​​of the matrix. According to the threshold value set in advance, when H>threshold, the pixel point is considered to be a dense feature point.

[0058] Step S303B: Use the KLT algorithm to track and match the dense feature points to obtain matching feature point pairs.

[0059] Specifically, it includes: using the KLT algorithm to track and match dense feature points between images, and comparing the square of the pixel grayscale difference in the preset window of the two foreground images to determine whether there is a matching feature point pair in the window; the implementation process is as follows: 1. Detect the feature points of the first picture.

[0060] 2. In the second picture, translate or map each feature point to estimate the motion trajectory.

[0061] 3. Track dense feature points based on motion trajectory.

[0062] Before feature tracking, a square feature window W with a side length of h is required around the dense feature points. The corresponding points of the same point in the two foreground images are defined as (ux, uy) and (ux+dx, uy+dy), where dx and dy are the directions of image movement, that is, the directions of object change. If the two points match, then in the window W, the foreground image A uses eh / 2 as the window, and the foreground image B uses e+h / 2 as the window, and the square difference of grayscale is the smallest. The offset of the dense feature point is d=[dx,dy]T, the projection point is e=[ux,uy]T, and the square difference of grayscale ε is expressed as follows according to the integral expression:

[0063] Among them, I(e) and J(e) are the grayscale values ​​on images I and J, that is, the grayscale values ​​on image 1 and image 2 respectively; w(e) is the function of window W. The partial derivative of the above formula is calculated in the direction of d, and when the result is 0, the minimum value of ε is obtained.

[0064] In a possible implementation, before extracting dense feature points in the foreground image, the method further includes: using an emphasize operator to enhance the image contrast of the foreground image.

[0065] Figure 5 The third flowchart of a method for displaying a power distribution panel based on computer vision provided in an embodiment of the present invention.

[0066] In a possible implementation, in step S304, mismatched feature points are removed from the matched feature points. Figure 5 As shown, steps S501-S504 are included.

[0067] Step S501 : selecting a preset number of sample feature points from the matching feature points, where the preset number of sample feature points are not collinear.

[0068] Step S502: construct a matrix model according to a preset number of sample feature points.

[0069] Exemplarily, four sample points are randomly selected from the set of matching feature point pairs, and the sample points cannot be collinear. Then, a transformation matrix is ​​calculated and recorded as a matrix model N.

[0070] Step S503: Calculate the projection error between each matching feature point and the matrix model.

[0071] Exemplarily, the distance between the matching point and the matrix model N is calculated, and the average of the calculated distances is obtained and set as the threshold Tr.

[0072] Step S504: if the projection error is greater than or equal to a preset error threshold, the matching feature points are removed as mismatched feature points.

[0073] Exemplarily, the projection error between all points in the matching feature point pair set and the model N is calculated, and if it is less than the threshold Tr, it is added to the feature point set Bn. If the projection error is greater than or equal to the preset error threshold, the matching feature point is removed as a mismatched feature point.

[0074] In an embodiment of the present invention, if the number of elements in Bn is greater than the optimal number Bn_best , then let Bn_best =Bn, and update the number of iterations at the same time , can be calculated by the following formula:

[0075] Wherein, p is the confidence level, which is generally 0.995, n is the ratio of the feature point set, and z is the number of sample points, which is 4. When the number of iterations is greater than kB, the process ends, otherwise the number of iterations increases by one, and the above steps S501-S504 are repeated.

[0076] Figure 6 A fourth flow chart of a method for displaying a power distribution panel based on computer vision provided in an embodiment of the present invention.

[0077] In a possible implementation, in step S103, the three-dimensional sparse point cloud is densified to obtain a dense point cloud, such as Figure 6 As shown, steps S601-S602 are included.

[0078] Step S601, obtaining depth maps of a multi-angle entity image at different angles.

[0079] In the embodiment of the present invention, a depth map is an image or an image channel, which contains the distance information of each point in the scene to the observation point, and is usually represented by grayscale, where brighter pixels represent objects closer to the observation point, and darker pixels represent objects farther away.

[0080] Step S602 , performing facet expansion and facet filtering on the three-dimensional sparse point cloud according to the image area information obtained from the depth map to obtain a dense point cloud.

[0081] Initialize matching features: extract feature points from the image, and for the feature points in the entity image at the first angle, find matching feature points in entity images at other angles to form matching point pairs, and use triangulation to form three-dimensional space points.

[0082] Patch expansion: Sort the three-dimensional space points formed above in ascending order according to the distance from the optical center O(I) of the solid image at the first angle, one by one until a patch is generated, and initialize the patch center cI(p), normal vector nI(p) and corresponding reference image RI(p); Patch filtering: Use three filters to filter out discontinuous patches that do not meet the consistency criteria.

[0083] Repeat the above steps to iteratively expand and filter the point cloud to finally obtain a dense point cloud.

[0084] In a possible implementation, in step S103, a static model of the power distribution panel is constructed based on the dense point cloud, including: constraining the reprojection error of the dense point cloud according to the light speed adjustment method, and performing surface reconstruction on the dense point cloud according to the Poisson surface reconstruction algorithm to obtain the static model of the distribution panel.

[0085] Specifically, based on the bundle adjustment theory to minimize the error, the Levebberg-Marquardt algorithm in the least squares method is used to reduce the reprojection error, as shown in the following formula:

[0086] Among them, K represents the number of spatial points, L represents the number of camera angles, Indicates Calculate the 3D point in The camera plane projection at angles, Represents a three-dimensional point in space To Camera The predicted projection point of the plane, d(x,y) represents the Euclidean distance between the image points x and y. Using the Poisson surface reconstruction algorithm, the dense point cloud is reconstructed to form a surface like Figure 7 The static model of the distribution panel is shown.

[0087] In a possible implementation, a method for displaying a power distribution panel based on computer vision provided by the present invention further includes: extracting sparse feature points in a foreground image of the power distribution panel using a SURF algorithm to obtain the spatial coordinates of a camera device.

[0088] Specifically, the SURF algorithm is used to extract sparse feature points in the foreground image, and for each sparse feature point, a 64-dimensional descriptor is formed based on the Harr wavelet feature; the specific implementation process is as follows: 1. Construct Gaussian scale space and calculate Hessian matrix Given a point (x, y) in the foreground image, the Hessian matrix In scale The specific expression is:

[0089] in Represents the foreground image at (x, y) and the Gaussian second-order differential The convolution of similar.

[0090] 2. Compare each pixel point processed by the Hessian matrix with the 26 points in the neighborhood of the two-dimensional image space and scale space, preliminarily locate the key points, and then filter out the key points with relatively weak energy and the key points with incorrect positioning, and select the final stable sparse feature points.

[0091] 3. Determine the main direction allocation of sparse feature points and generate a 64-bit descriptor of the feature points.

[0092] Then, the Euclidean distance between the sparse feature points of the foreground images is calculated, compared with the set threshold, and the feature point pairs that are less than the threshold are matched; specifically, the closest distance d1 and the second closest distance d2 between any two features in different images are calculated, and when the ratio of the closest distance d1 to the second closest distance d2 is less than a certain threshold, it is determined to be a match. The expression of distance L is:

[0093] in, , The i-th bit of the 64-bit descriptor of different features respectively.

[0094] Finally, based on the successfully matched sparse feature points (i.e., matching feature points), the five-point method is used to calculate the intrinsic matrix, obtain the camera extrinsic parameters, i.e., the camera pose, and then obtain the spatial coordinates of the camera device; the specific implementation steps are as follows: 1. Randomly select five pairs of matching feature points to calculate the eigenvalue matrix E, as shown in the following formula:

[0095] in, , and , are the coordinates of a matching feature point pair in the two foreground images respectively.

[0096] 2. Perform singular value decomposition on the intrinsic matrix E to calculate the camera's rotation matrix R and translation matrix T .

[0097] 3. By obtaining the matching feature points between the foreground images and combining the corresponding relationship between the coordinate systems, the world coordinates of the camera when shooting the image, that is, the spatial coordinates of the camera device, can be calculated. As shown in the following formula:

[0098] Where: M1 represents the intrinsic parameter matrix of the camera, which is obtained by Zhang Zhengyou calibration method; M2 represents the extrinsic parameter matrix of the camera, which is mainly obtained by obtaining the intrinsic matrix obtained above and performing singular value decomposition.

[0099] The following is an explanation of how to obtain the intrinsic parameter matrix of the camera in an embodiment of the present invention. In the embodiment of the present invention, multi-angle entity images can be input into the Camera Calibrator toolbox in the Matlab software to calculate and obtain the camera intrinsic parameters, namely, distortion parameters, camera focal length, etc., to obtain the camera intrinsic parameter matrix.

[0100] By adopting the above method, the present invention can achieve at least one of the following beneficial effects: 1. Through computer vision technology, a static model of the power distribution panel was constructed, and the real-time operation information of the power distribution panel was integrated into the static model of the power distribution panel, which improved the intuitiveness and efficiency of operation and maintenance management. The acquisition, processing and display of multi-angle physical images of the power distribution panel were realized, and the digital level of the power distribution panel was improved.

[0101] 2. By identifying, segmenting and extracting feature points of the distribution panel area in multi-angle solid images and eliminating mismatched feature points, the accuracy and quality of the three-dimensional sparse point cloud are improved, ensuring the accuracy and reliability of subsequent three-dimensional modeling.

[0102] 3. By utilizing the depth map information, the three-dimensional sparse point cloud is densified, further improving the detail richness of the distribution panel model.

[0103] 4. The light speed adjustment method and Poisson surface reconstruction algorithm are used to effectively constrain the reprojection error of dense point clouds, achieve high-precision surface reconstruction, and improve the accuracy and practicality of the static model of the distribution panel.

[0104] 5. Through the combination of Harris corner detection algorithm and KLT algorithm, the effective extraction and matching of dense feature points in the foreground image is realized, and the accuracy and stability of feature point pairs are improved.

[0105] 6. Before extracting dense feature points, the contrast of the foreground image is enhanced by the emphasize operator, which improves the image quality and further improves the accuracy and reliability of feature point extraction.

[0106] 7. By constructing a matrix model and calculating projection errors, the mismatched feature points are effectively eliminated, further improving the accuracy and quality of the three-dimensional sparse point cloud, providing a more reliable data foundation for subsequent three-dimensional modeling and operation and maintenance management.

[0107] The following is a description of the computer vision-based power distribution panel display device provided by the present invention. The computer vision-based power distribution panel display device described below and the computer vision-based power distribution panel display method described above can be referenced to each other.

[0108] Figure 8 A schematic diagram of the structure of a computer vision-based power distribution panel display device provided by the present invention includes: The image acquisition unit 810 is used to acquire multi-angle physical images of the power distribution panel.

[0109] The feature point matching unit 820 is used to extract matching feature points in the multi-angle entity image and map the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0110] The feature point processing unit 830 is used to densify the three-dimensional sparse point cloud to obtain a dense point cloud.

[0111] The model building unit 840 is used to build a static model of the power distribution panel according to the dense point cloud.

[0112] The information display unit 850 is used to display the operation information of the power distribution panel obtained in real time in the static model of the power distribution panel.

[0113] In a possible implementation, the feature point matching unit 820 includes: The area recognition subunit is used to recognize the distribution panel area in the multi-angle entity image.

[0114] The image segmentation subunit is used to segment the distribution panel area according to the GrabCut algorithm to obtain the foreground image of the power distribution panel.

[0115] The feature point matching subunit is used to extract dense feature points in the foreground image and perform feature point pair matching on the dense feature points to obtain matching feature point pairs.

[0116] The feature point elimination subunit is used to eliminate mismatched feature points in the matching feature point pairs.

[0117] The feature point mapping subunit is used to map the matching feature points excluding the mismatched feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0118] In a possible implementation, the feature point processing unit 830 includes: The depth map acquisition subunit is used to acquire the depth map of the multi-angle entity image at different angles.

[0119] The dense point cloud acquisition subunit is used to perform facet expansion and facet filtering on the three-dimensional sparse point cloud according to the image domain information obtained in the depth map to obtain a dense point cloud.

[0120] In a possible implementation, the model building unit 840 is specifically used to constrain the reprojection error of the dense point cloud according to the light speed adjustment method, and to perform surface reconstruction on the dense point cloud according to the Poisson surface reconstruction algorithm to obtain a static model of the distribution panel.

[0121] In a possible implementation, the feature point matching subunit is used to extract dense feature points in the foreground image using the Harris corner point detection algorithm; and to track and match the dense feature points using the KLT algorithm to obtain matching feature point pairs.

[0122] In a possible implementation manner, the feature point matching unit 820 further includes: an image enhancement subunit, configured to enhance the image contrast of the foreground image by using an emphasise operator.

[0123] In a possible implementation, the feature point elimination subunit is specifically used to select a preset number of sample feature points from the matching feature points, and the preset number of sample feature points are not collinear; construct a matrix model based on the preset number of sample feature points; calculate the projection error between each matching feature point and the matrix model; if the projection error is greater than or equal to a preset error threshold, the matching feature point is eliminated as a mismatched feature point.

[0124] In a possible implementation, the computer vision-based power distribution panel display device further includes: a camera coordinate acquisition unit for extracting sparse feature points in the foreground image of the power distribution panel using a SURF algorithm to obtain the spatial coordinates of the camera device.

[0125] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the power distribution panel display method based on computer vision, and the method includes: Get multi-angle solid images of the power distribution panel.

[0126] The matching feature points in the multi-angle entity images are extracted and mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0127] The three-dimensional sparse point cloud is densified to obtain a dense point cloud.

[0128] A static model of the power distribution panel is constructed based on the dense point cloud.

[0129] The real-time operation information of the power distribution panel is displayed in the static model of the distribution panel.

[0130] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0131] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the power distribution panel display method based on computer vision provided by the above methods, the method includes: Get multi-angle solid images of the power distribution panel.

[0132] The matching feature points in the multi-angle entity images are extracted and mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0133] The three-dimensional sparse point cloud is densified to obtain a dense point cloud.

[0134] A static model of the power distribution panel is constructed based on the dense point cloud.

[0135] The real-time operation information of the power distribution panel is displayed in the static model of the distribution panel.

[0136] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the computer vision-based power distribution panel display method provided by the above methods, the method comprising: Get multi-angle solid images of the power distribution panel.

[0137] The matching feature points in the multi-angle entity images are extracted and mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device.

[0138] The three-dimensional sparse point cloud is densified to obtain a dense point cloud.

[0139] A static model of the power distribution panel is constructed based on the dense point cloud.

[0140] The real-time operation information of the power distribution panel is displayed in the static model of the distribution panel.

[0141] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for displaying a power distribution panel based on computer vision, characterized in that: include: Use camera equipment to obtain multi-angle physical images of the power distribution panel; Extracting matching feature points in the multi-angle entity image, and mapping the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device; Densifying the three-dimensional sparse point cloud to obtain a dense point cloud; constructing a static model of the power distribution panel of the power distribution panel according to the dense point cloud; The operation information of the power distribution panel obtained in real time is displayed in the static model of the power distribution panel.

2. The method for displaying a power distribution panel based on computer vision according to claim 1, characterized in that: The extracting of matching feature points from the multi-angle entity image and mapping the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device includes: Identify the distribution panel area in the multi-angle entity image; Segment the distribution panel area according to the GrabCut algorithm to obtain a foreground image of the power distribution panel; Extracting dense feature points in the foreground image, and performing feature point pair matching on the dense feature points to obtain matching feature point pairs; Eliminating mismatched feature points in the matching feature point pairs; According to the spatial coordinates of the camera device, the matching feature points excluding the mismatched feature points are mapped into a three-dimensional sparse point cloud.

3. The method for displaying a power distribution panel based on computer vision according to claim 1, characterized in that: The step of densifying the three-dimensional sparse point cloud to obtain a dense point cloud includes: Obtain depth maps of multi-angle entity images at different angles; According to the image domain information obtained in the depth map, the three-dimensional sparse point cloud is subjected to patch expansion and patch filtering to obtain the dense point cloud.

4. The method for displaying a power distribution panel based on computer vision according to claim 1, characterized in that: The step of constructing a static model of a power distribution panel according to the dense point cloud comprises: The reprojection error of the dense point cloud is constrained according to the light speed adjustment method, and the dense point cloud is reconstructed according to the Poisson surface reconstruction algorithm to obtain the static model of the distribution panel.

5. The method for displaying a power distribution panel based on computer vision according to claim 2, characterized in that: The step of extracting dense feature points from the foreground image and performing feature point pair matching on the dense feature points to obtain matching feature point pairs includes: Using Harris corner detection algorithm to extract dense feature points in the foreground image; The KLT algorithm is used to track and match dense feature points to obtain matching feature point pairs.

6. The method for displaying a power distribution panel based on computer vision according to claim 2, characterized in that: Before extracting dense feature points in the foreground image, it also includes: The image contrast of the foreground image is enhanced using an emphasize operator.

7. The method for displaying a power distribution panel based on computer vision according to claim 2, characterized in that: The removing of mismatched feature points from the matching feature points comprises: Selecting a preset number of sample feature points from the matching feature points, wherein the preset number of sample feature points are not collinear; Constructing a matrix model according to the preset number of sample feature points; Calculating the projection error between each of the matching feature points and the matrix model; If the projection error is greater than or equal to a preset error threshold, the matching feature point is removed as a mismatched feature point.

8. The method for displaying a power distribution panel based on computer vision according to claim 1, further comprising: The SURF algorithm is used to extract sparse feature points in the foreground image of the power distribution panel to obtain the spatial coordinates of the camera device.

9. A computer vision-based power distribution panel display device, characterized in that: include: An image acquisition unit, used to acquire multi-angle physical images of the power distribution panel; A feature point matching unit, used to extract matching feature points in the multi-angle entity image, and map the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera device; A feature point processing unit, used for densifying the three-dimensional sparse point cloud to obtain a dense point cloud; A model building unit, used for building a static model of the power distribution panel of the power distribution panel according to the dense point cloud; The information display unit is used to display the operation information of the power distribution panel obtained in real time into the static model of the power distribution panel.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the computer vision-based power distribution panel display method as described in any one of claims 1 to 8 is implemented.

Images