Photovoltaic panel positioning method, apparatus, device and readable storage medium

Abstract

The application discloses a photovoltaic panel positioning method, device and equipment and a readable storage medium, relates to the technical field of image processing, and aims to improve the positioning accuracy of a photovoltaic panel. The method comprises the following steps: determining the GPS information of a first photovoltaic panel according to a power station map, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of a mobile device; the power station map is marked with the GPS information of a plurality of photovoltaic panels; starting from a position pointed to by the initial GPS information of the mobile device, acquiring a target image in the process of movement of the mobile device; determining the movement parameters of the mobile device according to the target image; and determining the GPS information of a second photovoltaic panel according to the movement parameters, the GPS information of the first photovoltaic panel and the power station map. The embodiment of the application can improve the positioning accuracy of a photovoltaic panel.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a photovoltaic panel positioning method, apparatus, device, and readable storage medium. Background Technology

[0002] GPS (Global Positioning System)-based positioning technology has been widely used; for example, it can be used to locate photovoltaic panels in power plant inspections. However, due to limitations in the deployment location of photovoltaic panels, in situations with complex terrain and weak GPS signals, conventional GPS positioning methods cannot accurately determine the location of the photovoltaic panels. Summary of the Invention

[0003] This application provides a positioning method, apparatus, device, and readable storage medium to improve the accuracy of photovoltaic panel positioning.

[0004] In a first aspect, embodiments of this application provide a photovoltaic panel positioning method, including:

[0005] Based on the power plant map, the GPS information of the first photovoltaic panel is determined, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device; the power plant map marks the GPS information of multiple photovoltaic panels;

[0006] Starting from the location indicated by the initial GPS information of the mobile device, the target image is acquired as the mobile device moves;

[0007] Based on the target image, determine the movement parameters of the mobile device;

[0008] The GPS information of the second photovoltaic panel is determined based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map.

[0009] Prior to determining the GPS information of the first photovoltaic panel, the method further includes:

[0010] Obtain test images of the photovoltaic panel;

[0011] Determine the width of a photovoltaic panel in the test image;

[0012] The scale is determined based on the width of a photovoltaic panel in the test image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera.

[0013] The test images include RGB images captured using the camera of the mobile device, and / or infrared images captured using an infrared imaging device connected to the mobile device.

[0014] The scale represents the ratio between the pixel size on the power plant map and the real world.

[0015] in,

[0016] Determining the movement parameters of the mobile device based on the target image includes:

[0017] Corner information of the photovoltaic panel is extracted from the current frame image and the previous frame image in the target image, respectively;

[0018] For the current frame image and the previous frame image, feature points of the current frame image and feature points of the previous frame image are extracted respectively within the image regions determined according to their respective corner point information;

[0019] Determine matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image;

[0020] Based on the matched feature point pairs, the movement parameters of the mobile device are determined.

[0021] The method further includes, after extracting feature points of the current frame image and feature points of the previous frame image within the image region determined according to their respective corner point information:

[0022] Extract key feature points from the feature points of the current frame image, and extract key feature points from the feature points of the previous frame image;

[0023] Determining matching feature point pairs from feature points in the current frame image and feature points in the previous frame image includes:

[0024] Determine matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image;

[0025] Determining the movement parameters of the mobile device based on the matched feature point pairs includes:

[0026] Based on the matched key feature point pairs, the movement parameters of the mobile device are determined.

[0027] The step of extracting feature points from the current frame image and the previous frame image within the image region determined based on their respective corner point information includes:

[0028] When the target image is an RGB image captured by the camera of the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the color histogram in the corner neighborhood.

[0029] When the target image is an infrared image captured by an infrared imaging device connected to the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the gradient histogram.

[0030] The process of determining matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image includes:

[0031] For a first feature point in the current frame image and each second feature point in the previous frame image, the angle between the first feature point and each second feature point is calculated using the cosine distance algorithm, where the first feature point is any feature point in the current frame image.

[0032] When the angle between the first feature point and the target second feature point meets a preset condition, the first feature point and the target second feature point are used as a matching feature point pair.

[0033] Wherein, when the target image includes both an RGB image and an infrared image, after determining the matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image, the method further includes:

[0034] The matching feature point pairs obtained using the RGB image and the matching feature point pairs obtained using the infrared image are fused to obtain the final matching feature point pairs.

[0035] Wherein, determining the movement parameters of the mobile device based on the matched feature point pairs includes:

[0036] The rotation and translation matrices are calculated using the coordinate values ​​of each matched feature point pair, and these matrices are used as the movement parameters.

[0037] The step of determining the GPS information of the second photovoltaic panel based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map includes:

[0038] The rotation and / or translation distance of the mobile device is obtained based on the rotation and translation matrix.

[0039] Based on the scale, the rotation and / or movement distance of the mobile device is converted into the rotation and / or translation distance on the power station map;

[0040] Based on the rotation and / or translation distance on the power station map and the GPS information of the first photovoltaic panel, the second photovoltaic panel is determined on the power station map, and the GPS information of the second photovoltaic panel is obtained;

[0041] The scale bar represents the ratio between the pixel size on the power station map and the real world.

[0042] Secondly, embodiments of this application provide a photovoltaic panel positioning device, comprising:

[0043] The first determining module is used to determine the GPS information of the first photovoltaic panel based on the power plant map, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device; the power plant map is marked with the GPS information of multiple photovoltaic panels.

[0044] The first acquisition module is used to acquire a target image starting from the location indicated by the initial GPS information of the mobile device and during the movement of the mobile device;

[0045] The second determining module is used to determine the movement parameters of the mobile device based on the target image;

[0046] The third determining module is used to determine the GPS information of the second photovoltaic panel based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map.

[0047] The device further includes:

[0048] The second acquisition module is used to acquire test images of the photovoltaic panel;

[0049] The fourth determining module is used to determine the width of a photovoltaic panel in the test image;

[0050] The fifth determining module is used to determine the scale based on the width of a photovoltaic panel in the test image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera.

[0051] The test images include RGB images captured using the camera of the mobile device, and / or infrared images captured using an infrared imaging device connected to the mobile device.

[0052] The scale represents the ratio between the pixel size on the power plant map and the real world.

[0053] The second determining module includes:

[0054] The first extraction submodule is used to extract the corner information of the photovoltaic panel from the current frame image and the previous frame image in the target image, respectively.

[0055] The second extraction submodule is used to extract feature points of the current frame image and feature points of the previous frame image respectively within the image regions determined according to their respective corner point information for the current frame image and the previous frame image.

[0056] The first determining submodule is used to determine a matching pair of feature points from the feature points of the current frame image and the feature points of the previous frame image;

[0057] The second determining submodule is used to determine the movement parameters of the mobile device based on the matched feature point pairs.

[0058] The second determining module further includes:

[0059] The third extraction submodule is used to extract key feature points from the feature points of the current frame image, and to extract key feature points from the feature points of the previous frame image.

[0060] The first determining submodule is used to determine matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image;

[0061] The second determining submodule is used to determine the movement parameters of the mobile device based on the matched key feature point pairs.

[0062] The second determining module further includes:

[0063] The third determining submodule is used to fuse the matching feature point pairs obtained using the RGB image and the matching feature point pairs obtained using the infrared image to obtain the final matching feature point pairs.

[0064] The first extraction submodule is used for:

[0065] When the target image is an RGB image captured by the camera of the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the color histogram in the corner neighborhood.

[0066] When the target image is an infrared image captured by an infrared imaging device connected to the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the gradient histogram.

[0067] The first determining submodule includes:

[0068] The first calculation unit is used to calculate the angle between the first feature point and each second feature point in the current frame image and each second feature point in the previous frame image using a cosine distance algorithm, wherein the first feature point is any feature point in the current frame image.

[0069] The first determining unit is used to determine the first feature point and the target second feature point as a matching feature point pair when the included angle between the first feature point and the target second feature point meets a preset condition.

[0070] The second determining submodule is used to calculate the rotation and translation matrix using the coordinate values ​​of each matched feature point pair, and to use the rotation and translation matrix as the movement parameter.

[0071] The third determining module includes:

[0072] The first acquisition submodule is used to obtain the rotation and / or translation distance of the mobile device based on the rotation and translation matrix;

[0073] The first conversion submodule is used to convert the rotation and / or movement distance of the mobile device into the rotation and / or translation distance on the power station map according to the scale.

[0074] The first determining submodule is used to determine the second photovoltaic panel on the power station map based on the rotation and / or translation distance on the power station map and the GPS information of the first photovoltaic panel, and to obtain the GPS information of the second photovoltaic panel;

[0075] The scale bar represents the ratio between the pixel size on the power station map and the real world.

[0076] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the photovoltaic panel positioning method described above.

[0077] Fourthly, embodiments of this application provide a readable storage medium for storing a program, characterized in that the program, when executed by a processor, implements the steps in the photovoltaic panel positioning method described above.

[0078] In this embodiment, the GPS information of the first photovoltaic panel is determined based on the initial GPS information of the mobile device, and the movement parameters of the mobile device are determined based on the acquired image. Then, the GPS information of the second photovoltaic panel is determined based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map. Therefore, using the solution of this embodiment, even in complex terrain and weak GPS signal conditions, the movement parameters of the mobile device can be determined based on the acquired target image, thereby locating the second photovoltaic panel corresponding to the current location of the mobile device. This reduces the impact of GPS signal strength on the positioning of the photovoltaic panel and improves the accuracy of photovoltaic panel positioning. Attached Figure Description

[0079] Figure 1 This is one of the flowcharts of the photovoltaic panel positioning method provided in the embodiments of this application;

[0080] Figure 2 This is a schematic diagram of matching feature point pairs in an embodiment of this application;

[0081] Figure 3 This is the second flowchart of the photovoltaic panel positioning method provided in the embodiments of this application;

[0082] Figure 4 This is one of the structural diagrams of the photovoltaic panel positioning device provided in the embodiments of this application;

[0083] Figure 5 This is the second structural diagram of the photovoltaic panel positioning device provided in the embodiments of this application;

[0084] Figure 6 This is one of the schematic diagrams of the second determining module in the embodiments of this application;

[0085] Figure 7 This is a second schematic diagram of the second determining module in the embodiments of this application;

[0086] Figure 8 This is the third schematic diagram of the second determining module in the embodiments of this application;

[0087] Figure 9 This is a schematic diagram of the first determining submodule in an embodiment of this application;

[0088] Figure 10 This is a schematic diagram of the third determining module in an embodiment of this application;

[0089] Figure 11 This is a schematic diagram of an electronic device in an embodiment of this application. Detailed Implementation

[0090] In the embodiments of this application, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0091] In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0092] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0093] See Figure 1 , Figure 1 This is a flowchart of a photovoltaic panel positioning method provided in an embodiment of this application. The method can be applied to mobile devices with cameras, such as mobile terminals and tablet computers. Figure 1 As shown, it includes the following steps:

[0094] Step 101: Determine the GPS information of the first photovoltaic panel based on the power station map, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device.

[0095] The power plant map includes GPS information for multiple photovoltaic panels. Of course, the map can also include GPS information for other objects or buildings. In this embodiment, the power plant map may be captured using a panoramic camera, and each photovoltaic panel in the map has corresponding GPS information (coordinates).

[0096] To determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in an image, a panoramic camera can be calibrated. Specifically, an RGB image of a photovoltaic panel captured by a mobile device's camera, or an infrared image of the photovoltaic panel captured by an infrared imaging device connected to the mobile device, or both, can be used to determine the width of a photovoltaic panel in the image. Then, based on the width of the photovoltaic panel in the image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera, a scale is determined. This scale represents the ratio between the pixel size on the power plant map and the real world. The camera parameters include the camera focal length *f*, the distance *z* from the camera to the photovoltaic panel, etc.

[0097] Specifically, the corner information of the photovoltaic panel can be obtained using a corner detection method for binary images, and the width w of the photovoltaic panel in the image can be obtained using the information of the four corner points. Then, the scale between the pixel size on the power plant map and the real world can be obtained using a similar triangle algorithm.

[0098]

[0099] Where scale represents the scale bar, f represents the camera focal length, z represents the distance from the camera to the photovoltaic panel, w represents the width of the photovoltaic panel in the image, and W represents the actual width of the photovoltaic panel.

[0100] Based on the current location of the mobile device, GPS information of the current location is obtained. Then, based on this GPS information, the first photovoltaic panel can be located on the power plant map, and the GPS information of the first photovoltaic panel is obtained. The GPS information of the current location of the mobile device can refer to the GPS information of the mobile device obtained through the device's positioning function or other means when executing the method of this application embodiment.

[0101] Step 102: Starting from the location indicated by the initial GPS information of the mobile device, acquire the target image as the mobile device moves.

[0102] Typically, GPS information can be used to determine the location. Therefore, in this step, the location pointed to by the initial GPS information of the mobile device can be determined using the initial GPS information of the mobile device; this location can also be referred to as the initial position of the mobile device. Thus, in this step, starting from the initial position of the mobile device, the target image is acquired as the mobile device moves.

[0103] The photovoltaic (PV) panels can be continuously photographed while the mobile device is in motion. Since PV panels are typically deployed in groups, different PV panels will be captured during the movement of the mobile device. The target image refers to an image of the PV panel captured during the movement of the mobile device; this image may include one or more PV panels, or it may not. The mobile device itself has a camera, and an infrared imaging device (such as an infrared camera) can also be connected to the mobile device. Therefore, the target image can be an RGB image of the PV panel captured by the mobile device's camera, or an infrared image of the PV panel captured by an infrared imaging device connected to the mobile device, or both.

[0104] When lighting conditions are good, RGB images can be used for subsequent processing; when ambient lighting is poor, infrared corner feature tracking of infrared images can be used to improve the accuracy of image tracking.

[0105] Step 103: Determine the movement parameters of the mobile device based on the target image.

[0106] During the movement of the mobile device, translation or rotation may be involved. Therefore, in this embodiment, the movement parameters can be represented by the rotation and translation attributes or parameters of the mobile device.

[0107] Each type of target image can be understood as comprising multiple image frames. Let's assume the currently acquired frame is called the current frame image, and the frame preceding the current frame image is called the previous frame image.

[0108] In this embodiment, firstly, the corner information of the photovoltaic panel can be extracted from the current frame image and the previous frame image in the target image. For example, the corner detection method can be used to obtain the four corner information of the photovoltaic panel, thereby determining the image region in the image that includes the photovoltaic panel.

[0109] Next, for the current frame image and the previous frame image, feature points of the current frame image and the previous frame image are extracted respectively within the image regions determined based on their respective corner point information. These feature points are used to determine the movement direction and distance of the mobile device during subsequent motion. Specifically, when the target image is an RGB image captured by the mobile device's camera, feature points of the current frame image and the previous frame image are extracted by calculating the color histogram within the corner neighborhood; when the target image is an infrared image captured by an infrared imaging device connected to the mobile device, feature points of the current frame image and the previous frame image are extracted by calculating the gradient histogram.

[0110] Then, matching feature point pairs are determined from the feature points of the current frame image and the feature points of the previous frame image.

[0111] When determining matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image, for each first feature point of the current frame image and each second feature point of the previous frame image, the angle between the first feature point and each second feature point is calculated using a cosine distance algorithm. Here, the first feature point can be any feature point in the current frame image, and the second feature point can be any feature point in the previous frame image. When the angle between the first feature point and the target second feature point meets a preset condition, the first feature point and the target second feature point are considered a matching feature point pair. Here, the target second feature point is a feature point among the second feature points.

[0112] Suppose that the ratios of two feature points are A (corresponding to vector a) and B (corresponding to vector b), the angle between the two feature points can be calculated using the following formula:

[0113]

[0114] Where θ represents the included angle. The smaller the included angle, the higher the similarity between the two feature points; the larger the included angle, the lower the similarity between the two feature points.

[0115] The preset conditions can be set as needed, for example, they can be a certain angle value. In this embodiment, two feature points are considered to match when the following conditions are met:

[0116]

[0117] In practical applications, when the target image includes both RGB and infrared images, matching feature point pairs obtained from the RGB and infrared images can be fused to obtain the final matched feature point pairs. Subsequent calculations of motion parameters are then based on these final matched feature point pairs. For example, {FRGB,(1-cos(θ)FIR}} can be used to match feature point pairs between the current image and the previous frame. Here, FRGB represents the feature point pair in the RGB image, FIR represents the feature point pair in the infrared image, and 1-cos(θ) represents the adaptive weight used to balance RGB and infrared features. This method improves the accuracy of the obtained matched feature point pairs.

[0118] Finally, the movement parameters of the mobile device are determined based on the matched feature point pairs.

[0119] When calculating the movement parameters, the rotation and translation matrices can be calculated using the coordinate values ​​of each matched feature point pair, and these matrices can be used as the movement parameters.

[0120] For example, such as Figure 2 As shown, the matching feature point pairs are (a1, a2), (b1, b2), (c1, c2), and (d1, d2). The rotation and translation matrices can then be calculated using the coordinates of each feature point as follows:

[0121]

[0122] Where [R|t] represents the rotation and translation matrix.

[0123] In the above process, to reduce the number of extracted feature points and ensure that the acquired variables are independent, dominant features can be extracted and features compressed to speed up subsequent processing. Therefore, in this embodiment, the PCA (Principal Component Analysis) method can be used to extract key feature points from the feature points of the current frame image and from the feature points of the previous frame image. Then, in subsequent processes, matching key feature point pairs can be determined from the key feature points of the current frame image and the previous frame image. Finally, the movement parameters of the mobile device are determined based on the matching key feature point pairs.

[0124] Step 104: Determine the GPS information of the second photovoltaic panel based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map.

[0125] As mentioned earlier, the photovoltaic panel captured will change as the mobile device moves. Here, the second photovoltaic panel refers to the photovoltaic panel displayed in the target image.

[0126] In this step, the rotation and / or translation distance of the mobile device can be obtained based on the rotation and translation matrix. The values ​​in the rotation and translation matrix represent rotation or translation distances. Then, based on the scale, the rotation and / or translation distance of the mobile device is converted into rotation and / or translation distances on the power plant map. Afterwards, based on the rotation and / or translation distances on the power plant map and the GPS information of the first photovoltaic panel, the second photovoltaic panel is determined on the power plant map, and its GPS information is obtained; wherein the scale is the aforementioned obtained scale.

[0127] In this embodiment, the GPS information of the first photovoltaic panel is determined based on the initial GPS information of the mobile device, and the movement parameters of the mobile device are determined based on the acquired image. Then, the GPS information of the second photovoltaic panel is determined based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map. Therefore, using the solution of this embodiment, even in complex terrain and weak GPS signal conditions, the movement parameters of the mobile device can be determined based on the acquired target image, thereby locating the second photovoltaic panel corresponding to the current location of the mobile device. This reduces the impact of GPS signal strength on the positioning of the photovoltaic panel and improves the accuracy of photovoltaic panel positioning.

[0128] See Figure 3 , Figure 3 This is a flowchart of a photovoltaic panel positioning method provided in an embodiment of this application. The method can be applied to mobile devices equipped with cameras, such as mobile terminals and tablets. The mobile device can also be connected to an infrared camera for acquiring infrared images. Figure 3 As shown, it includes the following steps:

[0129] Step 301: Obtain test images of the photovoltaic panel.

[0130] The test images include RGB images captured using the camera of the mobile device, and / or infrared images captured using an infrared imaging device connected to the mobile device.

[0131] Step 302: Determine the width of a photovoltaic panel in the test image.

[0132] Specifically, the information of the four corner points of the photovoltaic panel can be obtained by using the corner detection method of binary image, and the width w of the photovoltaic panel in the image can be obtained by using the information of the four corner points.

[0133] Step 303: Determine the scale based on the width of a photovoltaic panel in the test image, the actual width of a photovoltaic panel, and the camera parameters of the panoramic camera.

[0134] The scale represents the ratio between the pixel size on the power plant map and the real world. The power plant map includes GPS information for multiple photovoltaic panels. In this embodiment, the power plant map may be captured using a panoramic camera, and each photovoltaic panel in the map has corresponding GPS information (coordinates).

[0135] To determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in an image, a panoramic camera can be calibrated. Specifically, an RGB image of a photovoltaic panel captured by a mobile device's camera, or an infrared image of the photovoltaic panel captured by an infrared imaging device connected to the mobile device, or both, can be used to determine the width of a photovoltaic panel in the image. Then, based on the width of the photovoltaic panel in the image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera, a scale is determined. This scale represents the ratio between the pixel size on the power plant map and the real world. The camera parameters include the camera focal length *f*, the distance *z* from the camera to the photovoltaic panel, etc.

[0136] Specifically, the corner information of the photovoltaic panel can be obtained using a corner detection method for binary images, and the width w of the photovoltaic panel in the image can be obtained using the information of the four corner points. Then, the scale between the pixel size on the power plant map and the real world can be obtained using a similar triangle algorithm.

[0137]

[0138] Where scale represents the scale bar, f represents the camera focal length, z represents the distance from the camera to the photovoltaic panel, w represents the width of the photovoltaic panel in the image, and W represents the actual width of the photovoltaic panel.

[0139] Of course, in practical applications, if the scale is obtained through other means, steps 301 to 303 above can also be omitted.

[0140] Step 304: Determine the GPS information of the first photovoltaic panel based on the power station map, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device.

[0141] Step 305: Starting from the location indicated by the initial GPS information of the mobile device, acquire the target image as the mobile device moves.

[0142] Step 306: Determine the movement parameters of the mobile device based on the target image.

[0143] Step 307: Determine the GPS information of the second photovoltaic panel based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map.

[0144] Steps 304 to 307 can be referred to the description of steps 101 to 104 above.

[0145] In the previous embodiments of this application, a power plant map marked with GPS information was used to obtain the correspondence between the power plant map and real-world coordinates. Furthermore, a corner feature tracking method was employed using infrared and visible light images captured by infrared and visible light cameras, improving the accuracy of image tracking. Simultaneously, the image positioning method was improved by utilizing the repeatability of photovoltaic panels. Accurate GPS information can be obtained by continuously tracking images captured by a handheld device against the power plant map image.

[0146] In the above process, feature points from the obtained RGB and infrared images can be used to match and track the position of the photovoltaic panel using feature-based visual tracking and edge feature detection methods from the infrared images, thus locating the photovoltaic panel. Furthermore, using the solution of this application embodiment, even in complex terrain and weak GPS signal conditions, the movement parameters of the mobile device can be determined based on the acquired target image, thereby locating the second photovoltaic panel corresponding to the current position of the mobile device. This reduces the impact of GPS signal strength on the positioning of the photovoltaic panel and improves the accuracy of photovoltaic panel positioning.

[0147] This application provides a photovoltaic panel positioning device. The photovoltaic panel positioning device can be applied to a mobile device. For example, the photovoltaic panel positioning device can be part of the mobile device, or the photovoltaic panel positioning device can be the mobile device itself. See also... Figure 4 The device may include:

[0148] The first determining module 401 is used to determine the GPS information of the first photovoltaic panel based on the power plant map, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device; the power plant map marks the GPS information of multiple photovoltaic panels; the first acquiring module 402 is used to acquire a target image starting from the position pointed to by the initial GPS information of the mobile device during the movement of the mobile device; the second determining module 403 is used to determine the movement parameters of the mobile device based on the target image; the third determining module 404 is used to determine the GPS information of the second photovoltaic panel based on the movement parameters, the GPS information of the first photovoltaic panel, and the power plant map.

[0149] See Figure 5 The device further includes:

[0150] The second acquisition module 405 is used to acquire a test image of a photovoltaic panel; the fourth determination module 406 is used to determine the width of a photovoltaic panel in the test image; the fifth determination module 407 is used to determine a scale based on the width of a photovoltaic panel in the test image, the actual width of a photovoltaic panel, and the camera parameters of the panoramic camera; wherein, the test image includes an RGB image captured by the camera of the mobile device, and / or an infrared image captured by an infrared imaging device connected to the mobile device; the scale represents the ratio between the pixel size on the power plant map and the real world.

[0151] See Figure 6 The second determining module 403 includes:

[0152] The first extraction submodule 4031 is used to extract corner information of the photovoltaic panel from the current frame image and the previous frame image in the target image, respectively; the second extraction submodule 4032 is used to extract feature points of the current frame image and feature points of the previous frame image within the image area determined according to their respective corner information, respectively; the first determination submodule 4033 is used to determine matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image; the second determination submodule 4034 is used to determine the movement parameters of the mobile device according to the matching feature point pairs.

[0153] See Figure 7 The second determining module 403 further includes:

[0154] The third extraction submodule 4035 is used to extract key feature points from the feature points of the current frame image and to extract key feature points from the feature points of the previous frame image; the first determination submodule 4033 is used to determine matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image; the second determination submodule 4034 is used to determine the movement parameters of the mobile device based on the matching key feature point pairs.

[0155] The first extraction submodule is configured to: when the target image is an RGB image captured by the camera of the mobile device, extract feature points of the current frame image and feature points of the previous frame image by calculating the color histogram in the corner neighborhood; when the target image is an infrared image captured by an infrared imaging device connected to the mobile device, extract feature points of the current frame image and feature points of the previous frame image by calculating the gradient histogram.

[0156] See Figure 8The second determining module 403 further includes:

[0157] The third determining submodule 4036 is used to fuse the matching feature point pairs obtained using the RGB image and the matching feature point pairs obtained using the infrared image to obtain the final matching feature point pairs.

[0158] The second determining submodule 4034 is used to calculate the rotation and translation matrix using the coordinate values ​​of each matched feature point pair, and to use the rotation and translation matrix as the movement parameter.

[0159] See Figure 9 The first determining submodule 4033 includes:

[0160] The first calculation unit 40331 is used to calculate the angle between the first feature point and each second feature point using a cosine distance algorithm for the first feature point of the current frame image and each second feature point of the previous frame image, wherein the first feature point is any feature point in the current frame image; the first determination unit 40332 is used to determine the first feature point and the target second feature point as a matching feature point pair when the angle between the first feature point and the target second feature point meets a preset condition.

[0161] See Figure 10 The third determining module 404 includes:

[0162] The first acquisition submodule 4041 is used to obtain the rotation and / or translation distance of the mobile device according to the rotation and translation matrix; the first conversion submodule 4042 is used to convert the rotation and / or translation distance of the mobile device into the rotation and / or translation distance on the power station map according to the scale; the first determination submodule 4043 is used to determine the second photovoltaic panel on the power station map according to the rotation and / or translation distance on the power station map and the GPS information of the first photovoltaic panel, and obtain the GPS information of the second photovoltaic panel; wherein, the scale represents the scale between the pixel size on the power station map and the real world.

[0163] The apparatus provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0164] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] See Figure 11 This application also provides a hardware structure for an electronic device. For example... Figure 11 As shown, the electronic device 1100 includes:

[0167] Processor 1102; and

[0168] Memory 1104 stores program instructions, wherein when the processor executes the program instructions, the processor 1102 performs the following steps:

[0169] Based on the power plant map, the GPS information of the first photovoltaic panel is determined, wherein the GPS information of the first photovoltaic panel is the same as the initial GPS information of the mobile device; the power plant map marks the GPS information of multiple photovoltaic panels;

[0170] Starting from the location indicated by the initial GPS information of the mobile device, the target image is acquired as the mobile device moves;

[0171] Based on the target image, determine the movement parameters of the mobile device;

[0172] The GPS information of the second photovoltaic panel is determined based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map.

[0173] Furthermore, such as Figure 11 As shown, the electronic device 1100 may also include a network interface 1101, an input device 1103, a hard disk 1105, and a display device 1106.

[0174] The various interfaces and devices described above can be interconnected via a bus architecture. The bus architecture can include any number of interconnected buses and bridges. Specifically, various circuits representing one or more central processing units (CPUs), represented by processor 1102, and one or more memories, represented by memory 1104, are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. It is understood that the bus architecture is used to implement communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, all of which are well known in the art and will not be described in detail herein.

[0175] The network interface 1101 can be connected to a network (such as the Internet, local area network, etc.), receive data from the network, and save the received data to the hard disk 1105.

[0176] The input device 1103 can receive various instructions input by the operator and send them to the processor 1102 for execution. The input device 1103 may include a keyboard or a clicking device (e.g., a mouse, trackball, touchpad, or touchscreen).

[0177] The display device 1106 can display the results obtained by the processor 1102 executing instructions.

[0178] The memory 1104 is used to store programs and data necessary for the operation of the operating system, as well as intermediate results and other data during the calculation process of the processor 1102.

[0179] It is understood that the memory 1104 in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. The memory 1104 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0180] In some implementations, memory 1104 stores elements, executable modules or data structures, or subsets thereof, or extended sets thereof: operating system 11041 and application program 11042.

[0181] The operating system 11041 includes various system programs, such as a framework layer, a core library layer, and a driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 11042 includes various applications, such as a browser, used to implement various application functions. Programs implementing the methods of the embodiments of this application can be included in the application program 11042.

[0182] The photovoltaic panel positioning method disclosed in the above embodiments of the present invention can be applied to or implemented by the processor 1102. The processor 1102 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the photovoltaic panel positioning method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the processor 1102. The processor 1102 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of the hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1104. Processor 1102 reads the information in memory 1104 and completes the steps of the above method in conjunction with its hardware.

[0183] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0184] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0185] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0186] Obtain test images of the photovoltaic panel;

[0187] Determine the width of a photovoltaic panel in the test image;

[0188] The scale is determined based on the width of a photovoltaic panel in the test image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera.

[0189] The test images include RGB images captured using the camera of the mobile device, and / or infrared images captured using an infrared imaging device connected to the mobile device.

[0190] The scale represents the ratio between the pixel size on the power plant map and the real world.

[0191] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0192] Corner information of the photovoltaic panel is extracted from the current frame image and the previous frame image in the target image, respectively;

[0193] For the current frame image and the previous frame image, feature points of the current frame image and feature points of the previous frame image are extracted respectively within the image regions determined according to their respective corner point information;

[0194] Determine matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image;

[0195] Based on the matched feature point pairs, the movement parameters of the mobile device are determined.

[0196] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0197] Extract key feature points from the feature points of the current frame image, and extract key feature points from the feature points of the previous frame image;

[0198] Determine matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image;

[0199] Based on the matched key feature point pairs, the movement parameters of the mobile device are determined.

[0200] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0201] When the target image is an RGB image captured by the camera of the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the color histogram in the corner neighborhood.

[0202] When the target image is an infrared image captured by an infrared imaging device connected to the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the gradient histogram.

[0203] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0204] For a first feature point in the current frame image and each second feature point in the previous frame image, the angle between the first feature point and each second feature point is calculated using the cosine distance algorithm, where the first feature point is any feature point in the current frame image.

[0205] When the angle between the first feature point and the target second feature point meets a preset condition, the first feature point and the target second feature point are used as a matching feature point pair.

[0206] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0207] The matching feature point pairs obtained using the RGB image and the matching feature point pairs obtained using the infrared image are fused to obtain the final matching feature point pairs.

[0208] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0209] The rotation and translation matrices are calculated using the coordinate values ​​of each matched feature point pair, and these matrices are used as the movement parameters.

[0210] Specifically, when the program is executed by the processor 1102, it can also perform the following steps:

[0211] The rotation and / or translation distance of the mobile device is obtained based on the rotation and translation matrix.

[0212] Based on the scale, the rotation and / or movement distance of the mobile device is converted into the rotation and / or translation distance on the power station map;

[0213] Based on the rotation and / or translation distance on the power station map and the GPS information of the first photovoltaic panel, the second photovoltaic panel is determined on the power station map, and the GPS information of the second photovoltaic panel is obtained;

[0214] The scale bar represents the ratio between the pixel size on the power station map and the real world.

[0215] The electronic device provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0216] This application also provides a readable storage medium storing a program. When executed by a processor, this program implements the various processes of the photovoltaic panel positioning method embodiments described above and achieves the same technical effects. To avoid repetition, it will not be described again here. The readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0217] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0218] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0219] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for positioning a photovoltaic panel, characterized in that, include: Based on the power plant map and the GPS information of the initial location of the mobile device, the first photovoltaic panel corresponding to the initial location of the mobile device on the power plant map is determined, and the GPS information of the first photovoltaic panel is determined based on the power plant map, wherein the GPS information of the first photovoltaic panel is the same as the GPS information of the initial location of the mobile device; the power plant map marks the GPS information of multiple photovoltaic panels. Starting from the initial position of the mobile device, during the movement of the mobile device, a target image of the photovoltaic panel is acquired by the mobile device at its current position. Based on the target image, determine the movement parameters of the mobile device; Based on the movement parameters, the GPS information of the first photovoltaic panel, and the power station map, the current location of the mobile device is determined to be the second photovoltaic panel corresponding to the power station map, and the GPS information of the second photovoltaic panel is determined based on the power station map.

2. The method according to claim 1, characterized in that, Before determining the GPS information of the first photovoltaic panel, the method further includes: Obtain test images of the photovoltaic panel; Determine the width of a photovoltaic panel in the test image; The scale is determined based on the width of a photovoltaic panel in the test image, the actual width of the photovoltaic panel, and the camera parameters of the panoramic camera. The test images include RGB images captured using the camera of the mobile device, and / or infrared images captured using an infrared imaging device connected to the mobile device; the scale bar represents the ratio between the pixel size on the power plant map and the real world.

3. The method according to claim 1, characterized in that, Determining the movement parameters of the mobile device based on the target image includes: Corner information of the photovoltaic panel is extracted from the current frame image and the previous frame image in the target image, respectively; For the current frame image and the previous frame image, feature points of the current frame image and feature points of the previous frame image are extracted respectively within the image regions determined according to their respective corner point information; Determine matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image; Based on the matched feature point pairs, the movement parameters of the mobile device are determined.

4. The method according to claim 3, characterized in that, After extracting feature points of the current frame image and feature points of the previous frame image within the image region determined according to their respective corner point information, the method further includes: Extract key feature points from the feature points of the current frame image, and extract key feature points from the feature points of the previous frame image; Determining matching feature point pairs from feature points in the current frame image and feature points in the previous frame image includes: Determine matching key feature point pairs from the key feature points of the current frame image and the key feature points of the previous frame image; Determining the movement parameters of the mobile device based on the matched feature point pairs includes: Based on the matched key feature point pairs, the movement parameters of the mobile device are determined.

5. The method according to claim 3, characterized in that, The step of extracting feature points from the current frame image and the previous frame image within the image region determined based on their respective corner point information includes: When the target image is an RGB image captured by the camera of the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the color histogram in the corner neighborhood. When the target image is an infrared image captured by an infrared imaging device connected to the mobile device, feature points of the current frame image and feature points of the previous frame image are extracted by calculating the gradient histogram.

6. The method according to claim 3, characterized in that, Determining matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image includes: For a first feature point in the current frame image and each second feature point in the previous frame image, the angle between the first feature point and each second feature point is calculated using the cosine distance algorithm, where the first feature point is any feature point in the current frame image. When the angle between the first feature point and the target second feature point meets a preset condition, the first feature point and the target second feature point are used as a matching feature point pair.

7. The method according to claim 3 or 6, characterized in that, When the target image includes both RGB and infrared images, after determining matching feature point pairs from the feature points of the current frame image and the feature points of the previous frame image, the method further includes: The matching feature point pairs obtained using the RGB image and the matching feature point pairs obtained using the infrared image are fused to obtain the final matching feature point pairs.

8. The method according to claim 3, characterized in that, Determining the movement parameters of the mobile device based on the matched feature point pairs includes: The rotation and translation matrices are calculated using the coordinate values ​​of each matched feature point pair, and these matrices are used as the movement parameters.

9. The method according to claim 8, characterized in that, Based on the movement parameters, the GPS information of the first photovoltaic panel, and the power plant map, the current location of the mobile device is determined to correspond to the second photovoltaic panel on the power plant map, and the GPS information of the second photovoltaic panel is determined based on the power plant map, including: The rotation and / or translation distance of the mobile device is obtained based on the rotation and translation matrix. Based on the scale, the rotation and / or movement distance of the mobile device is converted into the rotation and / or translation distance on the power station map; Based on the rotation and / or translation distance on the power station map and the GPS information of the first photovoltaic panel, the second photovoltaic panel is determined on the power station map, and the GPS information of the second photovoltaic panel is obtained; The scale bar represents the ratio between the pixel size on the power station map and the real world.

10. A photovoltaic panel positioning device, characterized in that, include: The first determining module is used to determine the first photovoltaic panel corresponding to the initial position of the mobile device on the power station map based on the power station map and the GPS information of the initial position of the mobile device, and to determine the GPS information of the first photovoltaic panel based on the power station map, wherein the GPS information of the first photovoltaic panel is the same as the GPS information of the initial position of the mobile device; the power station map marks the GPS information of multiple photovoltaic panels. The first acquisition module is used to acquire a target image of the photovoltaic panel taken by the mobile device at its current position, starting from the initial position of the mobile device and during the movement of the mobile device. The second determining module is used to determine the movement parameters of the mobile device based on the target image; The third determining module is used to determine the second photovoltaic panel corresponding to the current location of the mobile device on the power station map based on the movement parameters, the GPS information of the first photovoltaic panel and the power station map, and to determine the GPS information of the second photovoltaic panel based on the power station map.

11. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the photovoltaic panel positioning method as described in any one of claims 1 to 9.

12. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the photovoltaic panel positioning method as described in any one of claims 1 to 9.

Images