Methods, devices and terminal equipment for power equipment inspection

Abstract

This application relates to the field of power equipment inspection technology, and provides a method, apparatus, and terminal equipment for power equipment inspection. The method includes: acquiring images of the power equipment to be inspected, and the velocity, angular velocity, and acceleration of the acquiring equipment in a world coordinate system during image acquisition; the acquired images are multiple discontinuous frames; based on the velocity, angular velocity, and acceleration of the acquiring equipment in the world coordinate system during image acquisition, obtaining the rotation matrix and translation matrix of the acquired images relative to the world coordinate system; smoothing the acquired images; based on the smoothed acquired images, obtaining the virtual coordinates of feature points in the acquired images, where the feature points are the power equipment to be inspected; and based on the rotation matrix, translation matrix, and virtual coordinates of the feature points, obtaining the real coordinates of the feature points, i.e., obtaining the real coordinates of the power equipment to be inspected. This application can improve the efficiency of power equipment inspection.

Description

Technical Field

[0001] This application belongs to the field of power equipment inspection technology, and in particular relates to power equipment inspection methods, devices and terminal equipment. Background Technology

[0002] Power transmission equipment is the infrastructure for transforming and converting power voltage and current, and is crucial for ensuring a safe and stable power supply. Most power companies in my country face problems such as low efficiency, high labor consumption, inadequate testing, and incomplete data statistics during equipment inspections. Currently, power companies are using AR (Augmented Reality) technology to assist manual inspections. By integrating wearable technologies, inspection equipment is integrated into mobile devices, improving inspection efficiency.

[0003] However, with large amounts of data transmission, the speed of AR overlay and spatial perception processing is significantly reduced, affecting the inspection efficiency of substation equipment. Summary of the Invention

[0004] This application provides a method, apparatus, and terminal equipment for power equipment inspection to improve inspection efficiency.

[0005] This application is achieved through the following technical solution:

[0006] In a first aspect, embodiments of this application provide a method for inspecting power equipment, including:

[0007] Acquire images of the power equipment to be tested, as well as the velocity, angular velocity, and acceleration of the equipment in the world coordinate system when acquiring the images. The acquired images are multiple discontinuous frames.

[0008] Based on the velocity, angular velocity, and acceleration of the acquisition device in the world coordinate system during image acquisition, the rotation matrix and translation matrix of the acquired image relative to the world coordinate system are obtained.

[0009] The acquired image is smoothed, and based on the smoothed acquired image, the virtual coordinates of the feature points of the acquired image are obtained. The feature points are the power equipment to be detected.

[0010] Based on the rotation matrix, translation matrix, and virtual coordinates of the feature points, the true coordinates of the feature points are obtained, which in turn yields the true coordinates of the power equipment to be tested.

[0011] In conjunction with the first aspect, some possible implementations involve obtaining the virtual coordinates of feature points in the acquired image based on the smoothed image, including:

[0012] The scale space is calculated based on the smoothed acquired image.

[0013] Based on the scale space, multiple first feature points are obtained;

[0014] Based on the scale space, translation matrix, and rotation matrix, the first central feature point is obtained;

[0015] Based on the central first feature point and multiple first feature points, the feature points of the acquired image and the virtual coordinates of the feature points of the acquired image are obtained.

[0016] In conjunction with the first aspect, in some possible implementations, the rotation matrix is ​​calculated using the following formula:

[0017]

[0018] in, Let be the rotation matrix of the i-th frame of the image, Exp be the formula for calculating the exponential mapping from the Lie algebra to the Lie group, and Δt be the time interval between two adjacent frames. To capture the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image, This refers to the angular velocity deviation parameter of the acquisition device when acquiring the i-th frame of the image.

[0019] In conjunction with the first aspect, in some possible implementations, the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image is calculated using the following formula:

[0020]

[0021] in, To determine the angular velocity of the acquisition device when acquiring the i-th frame of the image, The white Gaussian noise represents the angular velocity of the acquisition device when acquiring the i-th frame of the image.

[0022] In conjunction with the first aspect, in some possible implementations, the translation matrix is ​​calculated using the following formula:

[0023]

[0024] in, This refers to the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image. To determine the acceleration deviation parameter of the acquisition device when acquiring the i-th frame of the image, g W For gravity parameters, This refers to the velocity of the acquisition device in the world coordinate system when acquiring the i-th frame of the image. The translation matrix for acquiring the image in the i-th frame.

[0025] In conjunction with the first aspect, in some possible implementations, the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image is calculated using the following formula:

[0026]

[0027] in, The acceleration white Gaussian noise of the acquisition device during the acquisition of the i-th frame image is used. This represents the acceleration of the acquisition device when acquiring the i-th frame of the image.

[0028] In conjunction with the first aspect, in some possible implementations, the relationship between the virtual coordinates and the real coordinates of the feature points is as follows:

[0029]

[0030] Among them, (X) W ,Y W Z W ,1) represents the true coordinates of the feature point, (X C ,Y C Z C (1) represents the virtual coordinates of the feature point, (u,v) represents the coordinates of the feature point in the acquired image, and p x p represents the horizontal translation of the feature point in the acquired image. y f is the vertical translation of the feature point in the acquired image. x f is the focal length of the feature point in the horizontal direction of the acquired image. y The focal length of the feature point in the vertical direction in the acquired image. The rotation matrix for acquiring the image in the i-th frame. The translation matrix for acquiring the image in frame j.

[0031] Secondly, embodiments of this application provide a power equipment inspection device, comprising:

[0032] The acquisition module is used to acquire images of the power equipment under test, as well as the velocity, angular velocity and acceleration of the acquisition equipment in the world coordinate system when acquiring the images. The acquired images are multiple discontinuous frame images.

[0033] The matrix module is used to obtain the rotation matrix and translation matrix of the acquired image relative to the world coordinate system based on the velocity, angular velocity and acceleration of the acquisition device in the world coordinate system when acquiring the image;

[0034] The coordinate module is used to smooth the acquired image. Based on the smoothed acquired image, the virtual coordinates of the feature points of the acquired image are obtained. The feature points are the power equipment to be detected.

[0035] The positioning module is used to obtain the real coordinates of the feature points based on the rotation matrix, translation matrix, and virtual coordinates of the feature points, that is, to obtain the real coordinates of the power equipment to be detected.

[0036] Thirdly, embodiments of this application provide a terminal device, including: a processor and a memory, the memory being used to store a computer program, wherein the processor executes the computer program to implement the power equipment inspection method as described in any of the first aspects.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the power equipment inspection method as described in any of the first aspects.

[0038] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the power equipment inspection method described in any one of the first aspects.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0040] The beneficial effects of the embodiments of this application compared with the prior art are:

[0041] This application obtains the rotation and translation matrices of the acquired image by measuring the velocity, angular velocity, and acceleration of the acquisition device in the world coordinate system when acquiring the image. The rotation and translation matrices reflect the positional relationship when the image was acquired. Then, the position of the feature point can be obtained by using the positional relationship and the virtual coordinates of the feature point in the acquired image. Using the rotation and translation matrices to locate the feature point can reduce the computational load of the device, speed up data processing, and improve inspection efficiency.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram illustrating an application scenario of the power equipment inspection method provided in one embodiment of this application;

[0045] Figure 2 This is a schematic flowchart of a power equipment inspection method provided in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the structure of a power equipment inspection device provided in one embodiment of this application;

[0047] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0049] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0050] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0051] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0052] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0054] Power transmission equipment is the infrastructure for transforming and converting power voltage and current, and is crucial for ensuring a safe and stable power supply. Most power companies in my country face problems such as low efficiency, high labor consumption, inadequate testing, and incomplete data statistics during equipment inspections. Currently, power companies are using AR (Augmented Reality) technology to assist manual inspections. By integrating wearable technologies, inspection equipment is integrated into mobile devices, improving inspection efficiency.

[0055] However, with large amounts of data transmission, the speed of AR overlay and spatial perception processing is significantly reduced, affecting the inspection effect of power equipment.

[0056] Based on the above problems, the power equipment inspection method in this application embodiment,

[0057] For example, embodiments of this application can be applied to, for example... Figure 1 In the exemplary scenario shown, the image acquisition device 10 acquires an image of the power equipment to be inspected and sends the image to the power equipment inspection device 20. The power equipment inspection device 20 obtains the real coordinates of the power equipment to be inspected through the acquired image, which facilitates maintenance and repair by the staff.

[0058] Figure 2 This is a schematic flowchart of a power equipment inspection method provided in an embodiment of this application, with reference to... Figure 2 The detailed description of the inspection method for this power equipment is as follows:

[0059] Step 101: Acquire the image of the power equipment to be tested, as well as the velocity, angular velocity and acceleration of the equipment in the world coordinate system when acquiring the image.

[0060] For example, images are acquired using a monocular or binocular camera in an augmented reality (AR) device.

[0061] For example, the acquired images are multiple discontinuous frame images, with a time interval of 1 second between two adjacent frame images. The existence of the time interval can prevent the differences between the multiple discontinuous frame images from being too small to calculate the results.

[0062] Specifically, the AR device first acquires a video clip using a monocular or binocular camera. Assuming the video is 10 seconds long and operates at 30fps, when acquiring frame images, 15 frames are skipped every 15 frames, meaning a 10-second video can be processed into 10 non-contiguous frame images. To make the image edges sharper, 30 frames from the video are read and their Laplacian variance values ​​are calculated. The 30 frames with the largest variance values ​​are then written into the frame image; the larger the variance value, the sharper the resulting image.

[0063] Specifically, AR devices integrate an inertial measurement unit (IMU) attitude sensing module. The integrated design makes operation convenient and facilitates the simplification of inspection equipment. The IMU typically includes three single-axis accelerometers and three single-axis gyroscopes, which can measure the three-axis attitude angular velocity and acceleration of AR mobile devices, and transmit acceleration data and continuous image data frames to a remote backend server via a 5G module.

[0064] Step 102: Based on the velocity, angular velocity, and acceleration of the acquisition device in the world coordinate system during image acquisition, obtain the rotation matrix and translation matrix of the acquired image relative to the world coordinate system.

[0065] For example, the rotation matrix is ​​calculated using the following formula:

[0066]

[0067] in, Let be the rotation matrix of the i-th frame of the image, Exp be the formula for calculating the exponential mapping from the Lie algebra to the Lie group, and Δt be the time interval between two adjacent frames. To capture the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image, This refers to the angular velocity deviation parameter of the acquisition device when acquiring the i-th frame of the image.

[0068] For example, the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of image is calculated using the following formula:

[0069]

[0070] in, To determine the angular velocity of the acquisition device when acquiring the i-th frame of the image, The white Gaussian noise represents the angular velocity of the acquisition device when acquiring the i-th frame of the image.

[0071] For example, the translation matrix is ​​calculated using the following formula:

[0072]

[0073] in, This refers to the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image. To determine the acceleration deviation parameter of the acquisition device when acquiring the i-th frame of the image, g W For gravity parameters, This refers to the velocity of the acquisition device in the world coordinate system when acquiring the i-th frame of the image. The translation matrix for acquiring the image in the i-th frame.

[0074] For example, the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of image is calculated using the following formula:

[0075]

[0076] in, The acceleration white Gaussian noise of the acquisition device during the acquisition of the i-th frame image is used. This represents the acceleration of the acquisition device when acquiring the i-th frame of the image.

[0077] Step 103: Smooth the acquired image. Based on the smoothed acquired image, obtain the virtual coordinates of the feature points of the acquired image. The feature points are the power equipment to be detected.

[0078] For example, the process of smoothing the acquired image is as follows: Where M represents the image brightness, and div represents the divergence calculation. This represents gradient calculation, where c(x,y,t) is the transfer function.

[0079] For example, obtaining the virtual coordinates of feature points in the acquired image based on the smoothed acquired image includes: calculating the scale space based on the smoothed acquired image; obtaining multiple first feature points based on the scale space; obtaining the central first feature point based on the scale space, translation matrix, and rotation matrix; and obtaining the feature points of the acquired image and the virtual coordinates of the feature points of the acquired image based on the central first feature point and the multiple first feature points.

[0080] Specifically, the scale space is calculated using the following formula:

[0081] L i+1 =L i (I+τA(L i ))

[0082] Where I is the identity matrix, A(L) i) is the matrix of the image in dimension i, τ is the time step, and L is the scale space.

[0083] Specifically, after obtaining the scale space, if a point is the maximum or minimum value compared to the eight surrounding points and the eighteen neighboring points in the upper and lower scale spaces (a total of 26 points), then that point is a feature point of the image in that scale space, i.e., the first feature point.

[0084] Next, the magnitude and direction of the gradient at each first feature point are calculated. The magnitude of the gradient is obtained using the following formula: Where m(x,y) is the magnitude of the gradient at (x,y), and the direction of the gradient is calculated by the following formula: θ(x,y)=arctan{[L(x,y+1)-L(x,y-1)] / [L(x+1,y)-L(x-1,y)]}, where θ(x,y) is the direction of the gradient at (x,y).

[0085] After accurately determining the gradient value and direction of each feature point, a feature descriptor is generated from this data. This descriptor has 64 or 128 dimensions, centered on the first feature point X. j The remaining first feature points are searched within the local region centered on the image, and then the correct first feature points are matched as the feature points of the acquired image. That is, based on the central first feature point and multiple first feature points, the final feature points of the acquired image are obtained. The advantage of this approach is that it uses the central first feature point X... j By focusing on a localized region, the search scope is narrowed, the number of matching iterations is reduced, and ultimately, matching efficiency can be improved.

[0086] The first feature point X at the center j (x j ,y j ,z j The coordinates of () are calculated using the following formula: Among them, X i (x i ,y i ,z i ) is the first feature point X centered on j At the coordinates of the previous frame, Let be a rotation matrix. It is a translation matrix.

[0087] Step 104: Based on the rotation matrix, translation matrix, and virtual coordinates of the feature points, obtain the real coordinates of the feature points, that is, obtain the real coordinates of the power equipment to be detected.

[0088] For example, the relationship between the virtual coordinates and the real coordinates of a feature point is as follows:

[0089]

[0090] Among them, (X) W ,Y W Z W ,1) represents the true coordinates of the feature point, (X C ,Y C Z C (1) represents the virtual coordinates of the feature point, (u,v) represents the coordinates of the feature point in the acquired image, and p x p represents the horizontal translation of the feature point in the acquired image. y f is the vertical translation of the feature point in the acquired image. x f is the focal length of the feature point in the horizontal direction of the acquired image. y The focal length of the feature point in the vertical direction in the acquired image. The rotation matrix for acquiring the image in the i-th frame. The translation matrix for acquiring the image in frame j.

[0091] Specifically, in the real-world coordinate system, the coordinates of the feature point in the acquired image are (u, v), and the translation amounts are p. x and p y The focal length in the corresponding direction is f. x and f y The homogeneous coordinates of the feature points are (X... C ,Y C Z C If ,1), then the following condition is met:

[0092] Specifically, in some embodiments, 3ds Max is used to model AR virtual objects. Modeling parameters are determined by combining prior knowledge such as national standards and specifications for power equipment. Artificial scene construction elements are set based on common elements in real-world scenarios to simulate highly realistic virtual 3D text, equipment, and other models. Images are captured by AR mobile devices equipped with 5G terminals. After the image source is encoded and encrypted, the mobile device accesses the wireless access network via a base station and transmits the data to BSC / RNC edge cloud devices with MEC infrastructure via the 5G network. The signal is then transmitted to the core network, where network elements are carried by an IP bearer network. The bearer network transmits the image information to the server-side access network. The server obtains the image information transmitted through the channel, decrypts it, and obtains continuous frame images after source decoding. After calculating the real and virtual coordinates, the correspondence between the virtual object coordinates and the real space coordinates is transmitted to the AR device via 5G. The AR device transmits virtual 3D objects, such as transformer models and temperature discharge sensing parameter models, to the server via 5G. Finally, the virtual models are registered in real time in the real scene of power inspection to achieve a virtual-real fusion display effect, which facilitates the inspection by staff.

[0093] The above-mentioned power equipment inspection method obtains the rotation and translation matrices of the acquired image by measuring its velocity, angular velocity, and acceleration in the world coordinate system. These matrices reflect the positional relationship when the image was acquired. The position of the feature point can then be obtained by using this positional relationship and the virtual coordinates of the feature point in the acquired image. Using the rotation and translation matrices to locate the feature point reduces the computational load on the equipment, speeds up data processing, and improves inspection efficiency.

[0094] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] Corresponding to the power equipment inspection method described in the above embodiments, Figure 3 The diagram shows a structural block diagram of the power equipment inspection device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0096] See Figure 3 The power equipment inspection device in this application embodiment may include an acquisition module 301, a matrix module 302, a coordinate module 303, and a positioning module 304.

[0097] Optionally, the acquisition module 301 is used to acquire the collected images of the power equipment to be tested, as well as the velocity, angular velocity and acceleration of the acquisition equipment in the world coordinate system when acquiring the collected images.

[0098] For example, the acquired images are multiple discontinuous frame images.

[0099] Optionally, matrix module 302 is used to obtain the rotation matrix of the acquired image relative to the world coordinate system and the translation matrix in the world coordinate system based on the velocity, angular velocity and acceleration of the acquisition device in the world coordinate system when acquiring the image.

[0100] For example, the rotation matrix is ​​calculated using the following formula:

[0101]

[0102] in, Let be the rotation matrix of the i-th frame of the image, Exp be the formula for calculating the exponential mapping from the Lie algebra to the Lie group, and Δt be the time interval between two adjacent frames. To capture the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image, This refers to the angular velocity deviation parameter of the acquisition device when acquiring the i-th frame of the image.

[0103] For example, the angular velocity of the acquisition device relative to the inertial frame when acquiring the i-th frame of image is calculated using the following formula:

[0104]

[0105] in, To determine the angular velocity of the acquisition device when acquiring the i-th frame of the image, The white Gaussian noise represents the angular velocity of the acquisition device when acquiring the i-th frame of the image.

[0106] For example, the translation matrix is ​​calculated using the following formula:

[0107]

[0108] in, This refers to the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of the image. To determine the acceleration deviation parameter of the acquisition device when acquiring the i-th frame of the image, g W For gravity parameters, This refers to the velocity of the acquisition device in the world coordinate system when acquiring the i-th frame of the image. The translation matrix for acquiring the image in the i-th frame.

[0109] For example, the acceleration of the acquisition device relative to the inertial frame when acquiring the i-th frame of image is calculated using the following formula:

[0110]

[0111] in, The acceleration white Gaussian noise of the acquisition device during the acquisition of the i-th frame image is used. This represents the acceleration of the acquisition device when acquiring the i-th frame of the image.

[0112] Optionally, coordinate module 303 is used to smooth the acquired image and obtain the virtual coordinates of the feature points of the acquired image based on the smoothed acquired image. The feature points are the power equipment to be detected.

[0113] For example, obtaining the virtual coordinates of feature points in the acquired image based on the smoothed acquired image includes: calculating the scale space based on the smoothed acquired image; obtaining multiple first feature points based on the scale space; obtaining the central first feature point based on the scale space, translation matrix, and rotation matrix; and obtaining the feature points of the acquired image and the virtual coordinates of the feature points of the acquired image based on the central first feature point and the multiple first feature points.

[0114] Optionally, the positioning module 304 is used to obtain the real coordinates of the feature points based on the rotation matrix, translation matrix and virtual coordinates of the feature points, that is, to obtain the real coordinates of the power equipment to be detected.

[0115] For example, the relationship between the virtual coordinates and the real coordinates of a feature point is as follows:

[0116]

[0117] Among them, (X) W ,Y W Z W ,1) represents the true coordinates of the feature point, (X C ,Y C Z C (1) represents the virtual coordinates of the feature point, (u,v) represents the coordinates of the feature point in the acquired image, and p x p represents the horizontal translation of the feature point in the acquired image. y f is the vertical translation of the feature point in the acquired image. x f is the focal length of the feature point in the horizontal direction of the acquired image. y The focal length of the feature point in the vertical direction in the acquired image. The rotation matrix for acquiring the image in the i-th frame. The translation matrix for acquiring the image in frame j.

[0118] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0120] This application also provides a terminal device, see [link to relevant documentation] Figure 4 The terminal device 500 may include at least one processor 510 and a memory 520, the memory 520 being used to store a computer program 521. The processor 510 is used to call and run the computer program 521 stored in the memory 520 to implement the steps in any of the above method embodiments, for example... Figure 2 Steps 101 to 104 in the illustrated embodiment. Alternatively, when the processor 510 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of modules 301 to 304 are shown.

[0121] For example, computer program 521 may be divided into one or more modules / units, one or more of which are stored in memory 520 and executed by processor 510 to complete this application. The one or more modules / units may be a series of computer program segments capable of performing specific functions, which describe the execution process of the computer program in terminal device 500.

[0122] Those skilled in the art will understand that Figure 4 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0123] The processor 510 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0124] The memory 520 can be an internal storage unit of the terminal device or an external storage device, such as a plug-in hard drive, a smart media card (SMC), a secure digital card (SD), or a flash card. The memory 520 is used to store the computer program and other programs and data required by the terminal device. The memory 520 can also be used to temporarily store data that has been output or will be output.

[0125] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0126] The power equipment inspection method provided in this application embodiment can be applied to terminal devices such as computers, wearable devices, vehicle-mounted devices, tablet computers, laptop computers, and netbooks. This application embodiment does not impose any restrictions on the specific type of terminal device.

[0127] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the various embodiments of the power equipment inspection method described above.

[0128] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the various embodiments of the power equipment inspection method.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for inspecting power equipment, characterized by, The method comprises the following steps: acquiring a collection image of a power equipment to be detected and a speed, an angular velocity and an acceleration of a collection device in a world coordinate system when the collection image is collected, the collection image being a plurality of discontinuous frame images; obtaining a rotation matrix of the collection image relative to the world coordinate system and a translation matrix in the world coordinate system based on the speed, the angular velocity and the acceleration of the collection device in the world coordinate system when the collection image is collected; performing smoothing processing on the collection image, and obtaining virtual coordinates of feature points of the collection image based on the collection image after the smoothing processing, the feature points being the power equipment to be detected; obtaining real coordinates of the feature points based on the rotation matrix, the translation matrix and the virtual coordinates of the feature points, that is, obtaining real coordinates of the power equipment to be detected; The process of smoothing the collected image is: wherein, M is the luminance of the image, div represents the divergence calculation, represents the gradient calculation, is the conduction function; The method further comprises the following steps: calculating a scale space based on the collection image after the smoothing processing; obtaining a plurality of first feature points based on the scale space; obtaining a central first feature point based on the scale space, the translation matrix and the rotation matrix; obtaining the feature points of the collection image and the virtual coordinates of the feature points of the collection image based on the central first feature point and the plurality of first feature points; a relationship between the virtual coordinates and the real coordinates of the feature points is as follows: wherein, is a real coordinate of the feature point, is a virtual coordinate of the feature point, is a coordinate of the feature point in the captured image, is a translation of the feature point in the horizontal direction in the captured image, is a translation of the feature point in the vertical direction in the captured image, is a focal length of the feature point in the horizontal direction in the captured image, is a focal length of the feature point in the vertical direction in the captured image, is a rotation matrix of the first i frame captured image, is a translation matrix of the first j frame captured image.

2. The power equipment inspection method of claim 1, wherein, The rotation matrix is calculated by the following formula: wherein, is the first i is the rotation matrix of the frame capturing the image, Exp is the formula for the exponential map from Lie algebra to Lie group, Δ t is the time interval between the two adjacent frames, is the angular velocity of the capturing device relative to the inertial frame when capturing the first i is the angular velocity of the capturing device when capturing the frame, is the angular velocity bias parameter of the capturing device when capturing the first i is the angular velocity bias parameter of the capturing device when capturing the frame.

3. The power equipment inspection method of claim 2, wherein, The acquisition first i The angular velocity of the acquisition device relative to the inertial system when the frame acquires the image is calculated by the following formula: in, To collect the first i The angular velocity of the acquisition device during frame image acquisition. To collect the first i White Gaussian noise in the angular velocity of the acquisition device when capturing frames of images.

4. The power equipment inspection method of claim 2, wherein, The translation matrix is calculated by the following formula: wherein, is the acceleration of the capture device relative to the inertial frame at the time of capture of the first i frame of the image, is the acceleration of the capture device at the time of capture of the first i frame of the image, is the gravity parameter, is the velocity of the capture device in the world coordinate frame at the time of capture of the first i frame of the image, is the translation matrix for the first i frame of the image.

5. The power equipment inspection method of claim 4, wherein, The acquisition first i The acceleration of the acquisition device relative to the inertial system when the frame acquires the image is calculated by the following formula: wherein, for collecting the first i acceleration of the capturing device when capturing the image, for collecting the first i acceleration of the capturing device when capturing the image.

6. An electric power equipment inspection device characterized by comprising: The method comprises the following steps: an acquisition module is configured to acquire a collection image of a power equipment to be detected and a speed, an angular velocity and an acceleration of a collection device in a world coordinate system when the collection image is collected, the collection image being a plurality of discontinuous frame images; a matrix module is configured to obtain a rotation matrix of the collection image relative to the world coordinate system and a translation matrix in the world coordinate system based on the speed, the angular velocity and the acceleration of the collection device in the world coordinate system when the collection image is collected; a coordinate module is configured to perform smoothing processing on the collection image, and obtain virtual coordinates of feature points of the collection image based on the collection image after the smoothing processing, the feature points being the power equipment to be detected; a positioning module is configured to obtain real coordinates of the feature points based on the rotation matrix, the translation matrix and the virtual coordinates of the feature points, that is, obtain real coordinates of the power equipment to be detected; The coordinate module is specifically used for smoothing the collected image; the process of smoothing the collected image is: wherein, M is the brightness of the image, div represents a divergence calculation, represents a gradient calculation, is a conduction function; The coordinate module is further configured to: calculate a scale space based on the collection image after the smoothing processing; obtain a plurality of first feature points based on the scale space; obtain a central first feature point based on the scale space, the translation matrix and the rotation matrix; obtain the feature points of the collection image and the virtual coordinates of the feature points of the collection image based on the central first feature point and the plurality of first feature points; a relationship between the virtual coordinates and the real coordinates of the feature points is as follows: in, The true coordinates of the feature points. These are the virtual coordinates of the feature points. The coordinates of the feature point in the acquired image. The translation amount of the feature point in the horizontal direction in the acquired image. This represents the vertical displacement of the feature point in the acquired image. The focal length of the feature point in the horizontal direction in the acquired image. The focal length of the feature point in the vertical direction in the acquired image is given by [reference to focal length]. For the first i Rotation matrix of frame-acquired image, For the first j Translation matrix of the frame-captured image.

7. A terminal device comprising: a processor and a memory, the memory storing a computer program capable of running on the processor, and the processor implements the power equipment inspection method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program, when executed by a processor, implements the power equipment inspection method according to any one of claims 1 to 5.

Images