Photovoltaic construction component identification method and system

Abstract

The application relates to the technical field of construction drawing design, and specifically discloses a photovoltaic construction component identification method and system, which comprises the following steps: acquiring multi-modal data of a construction site under a unified time reference; mapping the multi-modal data to the same coordinate system to obtain site perception data that is synchronous in time and space; extracting more than one component candidate object of the construction site from the site perception data, determining a real scene component graph according to the correlation of the component candidate objects; acquiring a design component graph representing the correlation of design components based on a design reference model of the construction site, and determining the component correspondence relationship between the real scene component graph and the design component graph; and performing matching confidence evaluation according to the component correspondence relationship to determine an output component-level identification result. The multi-modal data is mapped to the same coordinate system to realize spatial consistency expression of information acquired by different sensors, the design component graph and the real scene component graph are established, and atlas matching is performed to improve the low-reliability matching result.

Description

Technical Field

[0001] This application relates to the field of construction drawing design technology, and in particular to a method and system for identifying photovoltaic construction components. Background Technology

[0002] Photovoltaic power plant construction sites typically contain a large number of repetitive components, which are distributed in an array in space, with the number of individual components ranging from tens of thousands to hundreds of thousands. During construction, it is necessary to accurately identify and verify the installation location, orientation, and numbering correspondence of each component to ensure construction quality and subsequent operation and maintenance traceability.

[0003] In scenarios with repetitive components, the object numbering and correspondence are prone to instability. In photovoltaic power plants, pile foundations, supports, and components are usually distributed repeatedly in a large-scale array. If existing methods rely solely on local visual features or local geometric shapes, problems such as component number mismatch, mismatch between adjacent components, identity drift under shading conditions, and inconsistency in the identity of the same component between multiple inspections can easily occur. Current recognition schemes often only output fuzzy or erroneous results. Summary of the Invention

[0004] The main purpose of this application is to provide a method and system for identifying photovoltaic construction components, aiming to solve the technical problem of low identification reliability in the context of large-scale photovoltaic power plant component scenarios in the prior art.

[0005] To achieve the above objectives, this application provides a method for identifying photovoltaic construction components, the method comprising the following steps: Acquire multimodal data of the construction site under a unified time reference; The multimodal data is mapped to the same coordinate system to obtain spatiotemporally synchronized field perception data; More than one candidate component from the construction site is extracted from the field perception data, and a real-scene component image is determined based on the correlation between the candidate components. Based on the design benchmark model of the construction site, obtain the design component diagram representing the relationship between the design components, and determine the component correspondence between the real-scene component diagram and the design component diagram; Based on the component correspondence, a matching confidence assessment is performed to determine the output component-level recognition result.

[0006] In one embodiment, the step of acquiring multimodal data of the construction site under a unified time reference includes: Obtain the minimum identification feature size of the component to be identified at the construction site and the number of target pixels covered in the image; The target ground resolution and target flight altitude are determined based on the minimum identification feature size and the number of target pixels covered. Based on the target ground resolution and the target flight altitude, the inspection task parameters are obtained, and based on the inspection task parameters, the multimodal data of each of the components to be identified at the construction site under a unified time reference are obtained.

[0007] In one embodiment, the multimodal data includes image data, thermal imaging data, positioning data, and attitude data. The step of mapping the multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data includes: The pose parameters at each acquisition time are determined based on the positioning data and attitude data. The image data and thermal imaging data are mapped to the same coordinate system based on the pose parameters to obtain spatiotemporally synchronized on-site perception data.

[0008] In one embodiment, the step of extracting more than one candidate component from the site perception data and determining the real-scene component image based on the association relationship of each candidate component includes: Based on the component category, candidate components are extracted from the image data and thermal imaging data, and the correlation between the candidate components is determined. Using each of the candidate component objects as nodes, edges are established between the corresponding nodes according to the association between each candidate component object to obtain the real-world component diagram.

[0009] In one embodiment, the step of obtaining design component drawings representing the relationships between design components based on a design benchmark model of the construction site, and determining the component correspondence between the actual component drawings and the design component drawings, includes: Design component drawings representing the relationships between design components are obtained based on the design benchmark model at the construction site. The association between the design component diagram and the actual component diagram is obtained, the topological difference term is calculated, and the matching cost is calculated. A bipartite graph is constructed using the design component and the candidate component as nodes on both sides and the matching cost as edge weights. The correspondence between the components is obtained by solving the bipartite graph.

[0010] In one embodiment, the step of evaluating the matching confidence based on the component correspondence and determining the output component-level recognition result includes: The matching confidence is calculated based on the matching cost, neighborhood consistency, and multimodal consistency corresponding to the component correspondence. When the matching confidence is lower than a preset threshold, the region where the candidate object of the component corresponding to the component is located is determined, and the reshoot task parameters are generated based on the region, the occlusion of the candidate object, and the image clarity. Based on the reshoot task parameters, multimodal data of the target area is reacquired until the matching confidence is not lower than the preset threshold, so as to determine the output component-level recognition result.

[0011] In one embodiment, the step of determining the output component-level recognition result includes: Based on the component correspondence, determine the number correspondence between each design component and the corresponding component candidate object; Based on the number correspondence, the positional and orientation deviations between the design component and the corresponding component candidate object are obtained to determine the output component-level recognition result.

[0012] Furthermore, to achieve the above objectives, this application also provides a photovoltaic construction component identification system, the system comprising: A multimodal acquisition device is used to acquire multimodal data from the construction site under a unified time reference. An edge analysis device is used to map the multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data; The edge analysis device is also used to extract more than one candidate component from the field perception data, and to determine the real-scene component image based on the correlation between the candidate components. The edge analysis device is also used to obtain design component diagrams representing the relationship between design components based on the design benchmark model of the construction site, and to determine the component correspondence between the real-scene component diagram and the design component diagram; The result interaction terminal is used to evaluate the matching confidence based on the correspondence of the components and determine the output component-level recognition result.

[0013] In one embodiment, the multimodal data includes: image data, thermal imaging data, positioning data, and attitude data, and the multimodal acquisition device includes: a flight platform; The flight platform is internally equipped with: a positioning unit, an image acquisition unit, a thermal imaging acquisition unit, an attitude acquisition unit, a synchronization triggering device, and a communication transceiver unit; The synchronization triggering device is connected to the first end of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit, respectively; the communication transceiver unit is connected to the second end of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit, respectively. The synchronization triggering device is used to transmit the synchronization triggering signal to the positioning unit, the image acquisition unit, the thermal imaging acquisition unit and the attitude acquisition unit respectively, so as to establish a unified time reference; The communication transceiver unit is used to send the positioning data acquired by the image data acquisition unit, the image data acquired by the image acquisition unit, the thermal imaging data acquired by the thermal imaging acquisition unit, and the attitude data acquired by the attitude acquisition unit to the edge analysis device.

[0014] In one embodiment, the multimodal acquisition device further includes: a local cache unit; The second ends of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit are respectively connected to the communication transceiver unit through the local cache unit.

[0015] The above-mentioned one or more technical solutions provided in this application may have the following advantages or at least achieve the following technical effects: This application discloses a method and system for identifying photovoltaic construction components. The method includes: acquiring multimodal data of the construction site under a unified time reference; mapping the multimodal data to the same coordinate system to obtain spatiotemporally synchronized site perception data; extracting more than one candidate component from the site perception data, and determining a real-scene component diagram based on the correlation between the candidate components; obtaining a design component diagram representing the correlation between the design components based on the design benchmark model of the construction site, and determining the component correspondence between the real-scene component diagram and the design component diagram; evaluating the matching confidence based on the component correspondence, and determining the output component-level identification result. By mapping multimodal data to the same coordinate system, spatial consistency of information acquired by different sensors is achieved. By extracting candidate components from the site perception data and establishing a real-scene component diagram, the correlation between on-site components can be expressed. Establishing a design component diagram and a real-scene component diagram and performing map matching improves the low-reliability matching results and maintains the stability of component identification results in large-scale repetitive array scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the photovoltaic construction component identification method of this application; Figure 2 This is a schematic diagram of the first process of the second embodiment of the photovoltaic construction component identification method of this application; Figure 3 This is a schematic diagram of the second process of the second embodiment of the photovoltaic construction component identification method of this application; Figure 4 This is a flowchart illustrating the third embodiment of the photovoltaic construction component identification method of this application; Figure 5 This is a flowchart illustrating the fourth embodiment of the photovoltaic construction component identification method of this application; Figure 6 This is a first structural schematic diagram of the photovoltaic construction component identification system of this application; Figure 7 This is a schematic diagram of the second structure of the photovoltaic construction component identification system of this application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a specific posture. If the specific posture changes, the directional indicators will also change accordingly.

[0021] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the use of "and / or" or "and / or" throughout the text includes three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution that simultaneously satisfies A and B. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0022] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a photovoltaic construction component identification system capable of achieving the above functions. The following description uses a photovoltaic construction component identification system as an example to illustrate this embodiment and the subsequent embodiments.

[0023] Based on this, the embodiments of this application provide a method for identifying photovoltaic construction components, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the photovoltaic construction component identification method of this application.

[0024] Step S10: Obtain multimodal data of the construction site under a unified time reference.

[0025] It should be understood that a unified time reference can be a time synchronization mechanism established by a synchronization triggering device, used to align the sampling times of different sensors to the same time series. Multimodal data can include various types of sensing data, including visible light image data, thermal imaging data, positioning data, and attitude data.

[0026] It should be noted that multimodal data acquisition under a unified time reference refers to data obtained by various sensors under the control of the same trigger signal or after time stamp alignment processing, so that different modal data can correspond to the same time or the same interpolation time.

[0027] Step S20: Map the multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data.

[0028] It should be understood that the same coordinate system can be the world coordinate system or the local engineering coordinate system, used to uniformly express the spatial information acquired by different sensors.

[0029] It should be noted that the process of mapping to the same coordinate system can be a coordinate transformation process achieved through the camera intrinsic parameter matrix, extrinsic parameter matrix, and pose transformation matrix.

[0030] Step S30: Extract more than one candidate component from the on-site perception data, and determine the real-scene component diagram based on the correlation between the candidate components.

[0031] It should be understood that candidate components can be suspected component targets identified and extracted from sensing data, including categories such as pile foundation candidates, support candidates, and component candidates. Relationships can be spatial adjacency, connection, row-column relationships, or regional subordination relationships between candidate components. The real-world component diagram can be a graph structure constructed with candidate components as nodes and relationships as edges.

[0032] It should be noted that "more than one candidate component" means that at least two candidate components are extracted from the on-site sensing data. Typically, hundreds to tens of thousands of candidate components can be extracted at a photovoltaic construction site. The correlation can be determined based on the spatial location, geometric shape, and physical connection relationships of the candidate components.

[0033] Step S40: Based on the design benchmark model of the construction site, obtain the design component drawings that represent the relationship between the design components, and determine the correspondence between the actual component drawings and the design component drawings.

[0034] It should be understood that the design baseline model can be a digital model built based on construction drawings, CAD models, or BIM models, containing information such as component numbers, spatial locations, orientation parameters, and adjacency relationships. The design component diagram can be a graph structure constructed from the design baseline model, where nodes represent design components and edges represent adjacency, connection, or region relationships between components. The component correspondence can be a mapping relationship between design components and candidate component objects, used to determine the i-th component in the design corresponds to the j-th component on site.

[0035] It should be noted that the correspondence between the design component diagram and the actual component diagram can be solved using a bipartite graph matching algorithm, the Hungarian algorithm, or a constrained optimization method.

[0036] Step S50: Evaluate the matching confidence based on the component correspondence and determine the output component-level recognition result.

[0037] It should be understood that the matching confidence level can be a quantitative assessment of the reliability of the component correspondence. The component-level identification result can include output information such as the component's existence status, component number correspondence, positional deviation, attitude deviation, or thermal feature binding result.

[0038] It should be noted that the matching confidence assessment can be calculated by comprehensively considering factors such as matching cost, neighborhood consistency, and multimodal consistency.

[0039] In one possible implementation, during the pile foundation construction phase, a multimodal acquisition terminal performs inspection and data collection on the pile location area. An edge analysis device designs a component diagram based on the designed pile location map, which includes the pile foundation design nodes and adjacent edges between adjacent piles. From the acquired visible light images, thermal images, RTK positioning data, and attitude data, candidate pile foundation objects are extracted using columnar feature detection and top edge recognition methods. Based on the spatial location of each candidate pile foundation object, the distance and directional relationships between adjacent piles are calculated to establish a real-scene component diagram. By calculating the positional differences, adjacency consistency, and row-column consistency between the designed pile foundation and the candidate pile foundations, the Hungarian algorithm is used to solve for the optimal correspondence, obtaining high-confidence matching results and low-confidence matching results. The system outputs a list of identified pile locations, information on suspected missing pile locations, and the specific location coordinates of low-confidence pile locations, and automatically generates re-shooting tasks for the low-confidence areas.

[0040] In another possible implementation, during the component deployment phase, the system identifies component boundaries and component arrays. An edge analysis device constructs a design component diagram of the component nodes based on the designed component layout diagram. Candidate component objects are identified from the field perception data using rectangular boundary detection and thermal distribution boundary extraction methods. A real-world component diagram is established using the row and column relationships between candidate components, the spacing relationships between adjacent components, and the thermal response boundaries. By combining location features, attitude features, scale features, thermal features, and row and column topology relationships, the matching cost between the designed components and candidate components is calculated, and the component correspondence is obtained. The system outputs the correspondence between each designed component number and the corresponding field candidate component, identifies suspected missing components, and accurately binds thermal anomaly features in the thermal imaging data to specific component numbers, providing basic data for hotspot tracing in the subsequent operation and maintenance phase.

[0041] It is easy to understand that this embodiment improves the problem of inaccurate binding of thermal features caused by the lack of strict synchronization between visible light images, thermal imaging data, and positioning attitude information by establishing a unified spatiotemporal synchronized multimodal acquisition mechanism. By mapping multimodal data to the same coordinate system, it helps to achieve spatial consistency in the expression of information acquired by different sensors. By extracting candidate components from the field perception data and building a real-scene component map, it is possible to express the correlation between field components, rather than processing individual components in isolation. By simultaneously building the design component map and the real-scene component map and performing map matching, the stability of object numbering is maintained in large-scale repetitive array scenarios, solving the problems of component number misalignment and adjacent component mismatch caused by relying solely on local geometric features. By introducing graph structure constraints such as adjacency, connectivity, and row-column relationships, the instability of matching caused by relying on geometric similarity is improved. Through the matching confidence evaluation mechanism, low-reliability matching results can be identified, providing a basis for subsequent re-shooting and verification, and helping to reduce mismatches and omissions caused by factors such as occlusion, reflection, and blurring.

[0042] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Please refer to Figure 2 , Figure 2 This is a schematic diagram of the first process of the second embodiment of the photovoltaic construction component identification method of this application. In this embodiment, step 10 includes: Step S201: Obtain the minimum identification feature size of the component to be identified at the construction site and the number of target pixels covered in the image.

[0043] It should be understood that the component to be identified can be any object on the construction site that needs to be identified and matched, including categories such as pile foundations, columns, beams, guide rails, and components. The minimum identification feature size can be the minimum geometric dimension of the key feature on the component to be identified used for identity determination or status recognition, such as the width of the top edge of a pile foundation, the line width of a component's border, or the cross-sectional width of a guide rail. The target pixel coverage number can be the minimum number of pixels that this minimum identification feature must cover in the image to ensure identification accuracy.

[0044] Step S202: Determine the target ground resolution and target flight altitude based on the minimum identification feature size and the number of target pixels covered.

[0045] It should be understood that the target ground resolution can be the ground sampling interval calculated based on the minimum recognizable feature size and the number of target pixels covered, representing the actual ground size corresponding to a unit pixel. The target flight altitude can be the UAV flight altitude calculated based on the camera focal length, pixel size, and target ground resolution.

[0046] It should be noted that the minimum identification feature size can be determined based on the identification requirements of different component categories. For example, for pile foundation identification, the minimum identification feature size can be set to the width of the top edge feature of the pile foundation; for component identification, the minimum identification feature size can be set to the component border line width or the junction box feature size. The target pixel coverage is usually set to at least 3 to 5 pixels to ensure feature distinguishability. The target ground resolution is calculated as follows: the target ground resolution is no greater than the ratio of the minimum identification feature size to the target pixel coverage. The target flight altitude can be calculated using a camera pinhole imaging model, by multiplying the focal length by the target ground resolution and then dividing by the pixel size.

[0047] Step S203: Obtain the inspection task parameters based on the target ground resolution and the target flight altitude, and acquire the multimodal data of each component to be identified at the construction site under a unified time reference based on the inspection task parameters.

[0048] It should be understood that inspection task parameters can include task configuration parameters such as flight altitude, flight path spacing, forward overlap rate, lateral overlap rate, flight speed, and shooting trigger interval. The generation of inspection task parameters can be determined by comprehensively considering factors such as the construction area range, component layout direction, obstacle distribution, and data acquisition time window.

[0049] In this embodiment, by pre-obtaining the minimum identification feature size of the component to be identified and setting the target pixel coverage number, sufficient spatial resolution is ensured in the collected multimodal data, mitigating the problem of insufficient or excessive image resolution caused by blindly setting the flight altitude. Calculating the target ground resolution based on the minimum identification feature size and the target pixel coverage number transforms the component identification requirement into a clear ground resolution indicator, determining the subsequent flight altitude. By calculating the target flight altitude based on camera parameters and the target ground resolution, flight efficiency is optimized while maintaining identification accuracy, avoiding limited acquisition area due to excessively low flight or loss of detail due to excessively high flight. By generating inspection task parameters including flight altitude and overlap rate, the data acquisition task is executed automatically, reducing the workload of manual parameter adjustment and avoiding data quality instability caused by relying on experience to set acquisition parameters.

[0050] Furthermore, multimodal data includes image data, thermal imaging data, positioning data, and attitude data. The generation of field perception data can be based on determining the pose parameters at each acquisition time according to the positioning data and attitude data; the image data and thermal imaging data are then mapped to the same coordinate system according to the pose parameters to obtain spatiotemporally synchronized field perception data.

[0051] It should be understood that image data can be RGB color or grayscale images acquired by a visible light camera, used to obtain visible light visual information of the construction site. Thermal imaging data can be temperature distribution images acquired by an infrared thermal imaging camera, used to obtain thermal characteristic information of the component surface. Positioning data can be three-dimensional spatial coordinates acquired by an RTK positioning unit, including east, north, and elevation coordinates. Attitude data can be attitude angle information acquired by a flight control attitude acquisition unit or inertial measurement unit, including roll, pitch, and yaw angles. Pose parameters can be six-degree-of-freedom pose information obtained by combining positioning and attitude data, used to describe the sensor's position and orientation in the world coordinate system. The process of mapping to the same coordinate system can be a coordinate transformation process achieved through camera intrinsic parameter calibration, extrinsic parameter calibration, and pose transformation. Spatiotemporally synchronized field perception data can be a unified digital expression obtained after time alignment and spatial registration of different modal data.

[0052] It should be noted that the acquisition time of image data can be recorded by the camera exposure timestamp, the acquisition time of thermal imaging data can be recorded by the sampling timestamp of the thermal imaging camera, the acquisition time of positioning data can be recorded by the timestamp of the RTK receiver, and the acquisition time of attitude data can be recorded by the timestamp of the flight control system. The pose parameters at each acquisition time can be determined by calculating the attitude parameters corresponding to the exposure time using linear interpolation or spline interpolation methods when the image exposure time and attitude sampling time are not perfectly aligned. The specific implementation of mapping to the same coordinate system can be as follows: first, convert the image pixel coordinates to camera normalized coordinates using the camera intrinsic parameter matrix; then, convert the normalized coordinates to world coordinates using the camera extrinsic parameter matrix and pose parameters. The mapping between image data and thermal imaging data can be performed independently based on their respective camera intrinsic and extrinsic parameters, or a dual-camera calibration can be performed to obtain the relative pose relationship before joint mapping. The same coordinate system is preferably the world coordinate system or a local engineering coordinate system to ensure that data acquired by different sensors at different times can be compared under a unified spatial reference.

[0053] In one possible implementation, for registration between thermal imaging data and image data, a dual-camera joint calibration method is used to pre-obtain the relative pose relationship between the visible light camera and the thermal imaging camera. During the data processing stage, the image data is first mapped to a world coordinate system to obtain a 3D point cloud. Then, based on the relative pose relationship between the two cameras, the 3D point cloud is projected onto the image plane of the thermal imaging camera, achieving pixel-level correspondence between the thermal imaging data and the image data. This method enables accurate projection of component boundaries identified in the image into the thermal imaging data, extraction of temperature distribution features in the corresponding areas, and accurate binding of thermal features to component numbers.

[0054] It is easy to understand that by determining the pose parameters at each acquisition moment based on positioning and attitude data, the accurate position and orientation of the sensor in space can be established, enabling subsequent coordinate mapping and data registration. By mapping image data and thermal imaging data to the same coordinate system according to pose parameters, a unified spatial representation of different modal data can be achieved, accurately associating the component objects identified in the visible light image with the temperature distribution in the thermal imaging data. By obtaining spatiotemporally synchronized field perception data, the complementarity of multimodal information can be used in subsequent component identification and matching processes to improve the identification instability caused by single-modal information being affected by occlusion, reflection, and changes in illumination.

[0055] Furthermore, referring to Figure 3 , Figure 3 This is a schematic diagram of the second process of the second embodiment of the photovoltaic construction component identification method of this application. Step S30 includes: Step S301: Extract candidate components from image data and thermal imaging data based on component categories, and determine the correlation between each candidate component.

[0056] It should be understood that component categories can be based on the function and form of components at the construction site, including pile foundations, supports, components, infrastructure, etc. Component candidate objects can be suspected component targets identified and extracted from image and thermal imaging data; each candidate object can have its location, orientation, scale, category confidence level, and other attributes recorded. Relationships can be spatial adjacency, geometric connection, row / column alignment, or regional subordination relationships between component candidate objects.

[0057] It should be noted that the process of extracting candidate objects based on component categories can employ corresponding detection algorithms for different categories. For example, a column feature detection algorithm can be used for pile foundations, a rectangular bounding box detection algorithm can be used for components, and a pole structure detection algorithm can be used for supports. Candidate objects extracted from image data can be identified based on visible light visual features, while candidate objects extracted from thermal imaging data can be identified based on temperature distribution boundaries. These two methods can be extracted independently and then fused, or they can be extracted jointly.

[0058] Step S302: Using each candidate component as a node, establish edges between the corresponding nodes according to the relationship between each candidate component to obtain a real-world component diagram.

[0059] It should be understood that nodes can be the basic units in a graph structure, with each node corresponding to a candidate component object. Node attributes can include the geometric parameters, thermal characteristic parameters, and category label of the candidate object. Edges can be expressions of the relationship between two nodes, and edge attributes can include the distance between the two candidate objects, orientation angle, connection type, etc. A real-world component graph can be a graph structure composed of a set of nodes and a set of edges, used to express the organizational and topological relationships of candidate components in the field.

[0060] It should be noted that the relationships between candidate components can be determined based on their spatial positions. For example, if the distance between two candidate components is less than a set threshold, they are considered adjacent; if their geometric shapes show connection features, they are considered connected; and if multiple candidate components are arranged linearly or in a matrix, they are considered row-column related. Establishing edges can involve adding an edge between two related nodes, with the edge weight assigned based on the strength of the relationship or distance. The construction of the real-world component graph can begin by establishing the entire node set, then traversing node pairs and determining relationships, and finally adding corresponding edges to form a complete graph structure.

[0061] In this embodiment, candidate components are extracted from image and thermal imaging data based on component categories. Corresponding recognition methods are employed for the morphological features of different component categories, addressing the issue of uneven recognition accuracy caused by using a unified detection algorithm. By simultaneously utilizing image and thermal imaging data, complementary recognition combining visible light and thermal features is possible, maintaining recognition stability under complex conditions such as occlusion and reflection. Determining the relationships between candidate components allows for the expression of spatial organizational relationships between components, enabling object-level matching using these relationships as constraints. By establishing edges between corresponding nodes and using each candidate component as a node to obtain a real-world component graph, the distribution of components on-site is transformed into a graph structure. Graph theory algorithms are then used for graph matching, avoiding the instability caused by isolated processing of individual components while ignoring inter-component relationships.

[0062] Based on the above embodiments of this application, in the third embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the photovoltaic construction component identification method of this application. In this embodiment, step 40 includes: Step S401: Obtain design component drawings representing the relationship between design components based on the design benchmark model of the construction site.

[0063] It should be understood that the design baseline model can be a digital model built based on construction drawings, BIM models, and pile layout drawings, containing information such as the number, category, location, orientation, and adjacency and connection relationships of each design component. The design component relationships can be adjacency, connection, row / column, or region-based relationships between design components, used to express the component organization method in the design intent. The design component diagram can be a graph structure with design components as nodes and design component relationships as edges.

[0064] It should be noted that the process of constructing the design component drawing can be as follows: first, read the spatial location and category information of each design component from the design baseline model, and then determine the relationship between the design components according to the design rules. For example, when the distance between two design components is less than the design spacing threshold, it is determined that there is an adjacency relationship. When the design drawing clearly indicates that there is a connection between two components, it is determined that there is a connection relationship.

[0065] Step S402: Obtain the relationship between the design component diagram and the actual component diagram, calculate the topology difference item, and calculate the matching cost.

[0066] It should be understood that the topological difference term can be used to measure the differences in graph structure between the design component diagram and the actual component diagram, including differences in node degree, adjacency consistency, and connectivity consistency. The matching cost can be a cost calculated by comprehensively considering the geometric differences, thermal feature differences, and topological differences between the design component and the candidate component, and is used to quantify the rationality of the matching.

[0067] It should be noted that the calculation of the topology difference term can be based on the difference between the number of adjacent components of design component i in the design component graph and the number of adjacent objects of candidate component j in the real-world component graph, or it can be based on the intersection ratio between the candidate objects corresponding to the adjacent components of design component i and the adjacent objects of candidate component j. The matching cost can be calculated using a weighted summation method, combining the position difference term, attitude difference term, scale difference term, thermal feature difference term, and topology difference term. The weights of each term can be set according to the task requirements.

[0068] Step S403: Construct a bipartite graph with the design component and component candidate objects as the two-sided nodes and the matching cost as the edge weight, and solve the bipartite graph to obtain the component correspondence.

[0069] It should be understood that a bipartite graph can be a graph structure where the nodes are divided into two sets: one side is the set of design components, and the other side is the set of candidate components. The weight of the edges represents the matching cost. Solving a bipartite graph can be done using a minimum-cost matching algorithm, the Hungarian algorithm, or a linear assignment algorithm to find the correspondence that minimizes the total cost. The component correspondence can be a one-to-one mapping between design components and candidate components, indicating that a certain component in the design corresponds to a certain candidate component in the field.

[0070] It should be noted that the construction of a bipartite graph can be achieved by calculating the matching cost between each design component in the set of design components and each candidate object in the set of candidate components, and using this cost as the edge weight to form a weighted complete bipartite graph or a sparse bipartite graph. Solving the bipartite graph can be done using the Hungarian algorithm or the KM algorithm to find the correspondence that minimizes the total matching cost. The solution result is a unique candidate object for each design component, or some design components may correspond to empty objects to indicate missing components.

[0071] It is easy to understand that this embodiment obtains design component diagrams based on a design baseline model, transforming design intent into a graph structure representation and utilizing design-side relational constraints for matching. By calculating topological difference terms based on the relationships between the design component diagrams and the actual component diagrams, graph structure constraints are introduced into the matching process, reducing numbering errors and mismatches in large-scale repetitive array scenarios. The matching cost, including the topological difference term, is calculated, comprehensively considering geometric features, thermal features, and relational relationships to improve matching accuracy and consistency. A bipartite graph is constructed using design components and candidate component objects as nodes on both sides and the matching cost as edge weights, transforming the component correspondence problem into a standard bipartite graph matching problem, which is efficiently solved using mature graph theory algorithms. By solving the bipartite graph, the component correspondence is obtained, determining the corresponding candidate on-site component for each design component, achieving component-level identity recognition and state comparison.

[0072] Based on the above embodiments of this application, in the fourth embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 , Figure 5 This is a flowchart illustrating the fourth embodiment of the photovoltaic construction component identification method of this application. In this embodiment, step 50 includes: Step S501: Calculate the matching confidence based on the matching cost, neighborhood consistency, and multimodal consistency corresponding to the component correspondence.

[0073] It should be understood that the matching cost can be a comprehensive measure of the geometric, thermal, and topological differences between the designed component and its candidate components; a lower matching cost indicates a more reasonable match. Neighborhood consistency refers to the degree of coordination between the corresponding component and its neighboring components. For example, when a designed component corresponds to a candidate component and its adjacent components also correspond to the adjacent components of that candidate component, the neighborhood consistency is high. Multimodal consistency refers to the degree of agreement between the visible light image recognition result and the thermal imaging recognition result. For example, when the boundary of the component identified in the image aligns with the temperature boundary identified in the thermal image, the multimodal consistency is high. Matching confidence is a reliability score calculated by combining the matching cost, neighborhood consistency, and multimodal consistency, used to quantify the credibility of the component correspondence.

[0074] It should be noted that the matching confidence score can be calculated using a weighted combination method. For example, the matching confidence score equals the position consistency score multiplied by a weight coefficient, plus the attitude consistency score multiplied by a weight coefficient, plus the topology consistency score multiplied by a weight coefficient, plus the thermal feature consistency score multiplied by a weight coefficient, plus the neighborhood continuity score multiplied by a weight coefficient. The position consistency score can be derived from the position difference item; the smaller the position difference, the higher the score. The topology consistency score can be derived from the topology difference item; the smaller the topology difference, the higher the score. The thermal feature consistency score can be derived from the thermal feature difference item; the smaller the thermal feature difference, the higher the score. The neighborhood continuity score can be calculated based on neighborhood consistency; when all adjacent components in the corresponding component relationship are valid matches, the neighborhood continuity score is high.

[0075] Step S502: When the matching confidence is lower than the preset threshold, determine the area where the candidate object of the component involved in the component correspondence is located, and generate the reshoot task parameters based on the area, the occlusion of the candidate object, and the image clarity.

[0076] It should be understood that the preset threshold can be a lower limit of confidence set according to the task accuracy requirements. When the matching confidence is lower than this threshold, the reliability of the correspondence is deemed insufficient. The region where the candidate component is located can be the spatial range of the candidate component or a certain buffer area around it. Occlusion can be the degree to which the candidate component is partially occluded by other objects in the image. Image sharpness can be a quantitative indicator of the image's sharpness, contrast, or blurriness. The reshoot task parameters can include parameters such as the coordinate frame of the reshoot area, suggested flight altitude, suggested shooting angle, overlap rate, and number of reshoots.

[0077] It should be noted that the preset threshold can be determined based on historical data statistics or task accuracy requirements. The generation of reshoot task parameters can be based on determining the coordinate frame of the reshoot area according to the position of the candidate low-confidence component, determining whether the shooting angle needs to be adjusted according to the occlusion situation, and determining whether the flight altitude needs to be reduced or the overlap rate increased according to the image clarity.

[0078] Step S503: Reacquire multimodal data of the target area according to the reshoot task parameters until the matching confidence is not lower than the preset threshold, so as to determine the output component-level recognition result.

[0079] It should be understood that the multimodal data of the target area can be image data, thermal imaging data, positioning data, and attitude data re-acquired for the re-shot area. The component-level recognition result can be the final recognition conclusion output after confidence verification, including the component's existence status, component number correspondence, position deviation, attitude deviation, and thermal feature binding result.

[0080] It should be noted that the execution of the re-shooting task can involve sending the re-shooting task parameters to the multimodal acquisition terminal, which then re-flies and collects data from the target area according to the re-shooting parameters. The processing after re-acquiring the data can involve fusing the newly acquired multimodal data with the original data, re-extracting candidate components, recalculating the matching cost, resolving the component correspondences, and re-evaluating the matching confidence. The termination condition until the matching confidence is not lower than a preset threshold can be when the matching confidence of all component correspondences reaches or exceeds the preset threshold, or when the number of re-shooting attempts reaches a set upper limit, at which point the final component-level recognition result is output.

[0081] The process is easy to understand: by calculating the matching confidence based on matching cost, neighborhood consistency, and multimodal consistency, the reliability of component correspondence is comprehensively evaluated based on multiple factors, identifying low-reliability matching results. By setting a preset threshold and triggering a re-shooting process when the matching confidence falls below the threshold, low-reliability results are avoided as the final output, thus improving recognition accuracy. By determining the region where the low-confidence component candidate object is located and generating re-shooting task parameters based on occlusion and image clarity, data in the problem area is re-acquired, improving data quality. By re-acquiring multimodal data of the target area based on the re-shooting task parameters and iteratively executing the process until the matching confidence is not lower than the preset threshold, a closed-loop verification mechanism is formed to maintain recognition stability under complex conditions such as occlusion, reflection, and blur.

[0082] Furthermore, based on the component correspondence, the number correspondence between each design component and its corresponding candidate component can be determined. The positional and orientation deviations between the design component and its corresponding candidate component are then obtained based on this number correspondence to determine the output component-level recognition result.

[0083] It should be understood that the numbering correspondence can be a mapping between the design number of a design component and the field identifier or extraction number of the component candidate object, used to clarify that a certain numbered component in the design corresponds to a specific object in the field. Position deviation can be the difference between the target position coordinates of the design component and the actual position coordinates of the corresponding component candidate object, which can be represented as a three-dimensional vector or deviation magnitude. Attitude deviation can be the difference between the target attitude parameters of the design component and the actual attitude parameters of the corresponding component candidate object, which can be represented as roll angle deviation, pitch angle deviation, and yaw angle deviation. The component-level identification result can be a complete output including the numbering correspondence, position deviation, attitude deviation, and optional scale deviation and thermal feature binding information.

[0084] It should be noted that the identification of the number correspondence can be directly obtained from the component correspondence. That is, for each valid component correspondence, the number of the design component and the identifier of the corresponding candidate component are extracted, and a number mapping table is established. The position deviation is calculated by subtracting the actual centroid coordinates of the candidate component from the target centroid coordinates of the design component, resulting in the eastward deviation, northward deviation, and elevation deviation. The position deviation modulus can be the square root of the sum of the squares of the deviations in the three directions. The attitude deviation is calculated by inversely transforming the target attitude matrix of the design component and the actual attitude matrix of the candidate component, decomposing them into roll angle deviation, pitch angle deviation, and yaw angle deviation. The position deviation and attitude deviation can be obtained by traversing all valid component correspondences and calculating the deviation of each component one by one. The output form of the component-level identification results can be a structured data table, a visualized deviation heatmap, or exported as interface data to the construction management system.

[0085] In essence, by determining the numbering correspondence based on the component correspondence, and mapping the design number to the on-site object, component-level traceability is achieved. Positional and orientation deviations are obtained based on the numbering correspondence, quantitatively assessing construction quality, identifying components with excessive installation errors, and guiding on-site rectification and adjustment. By outputting complete component-level identification results, accurate digital foundational data is provided for construction management, quality acceptance, and subsequent operation and maintenance, improving construction accuracy and traceability.

[0086] This application also provides a photovoltaic construction component identification system; please refer to... Figure 6 , Figure 6 This is a first structural schematic diagram of the photovoltaic construction component identification system of this application. The photovoltaic construction component identification system includes: Multimodal acquisition device 10 is used to acquire multimodal data of the construction site under a unified time reference; Edge analysis device 20 is used to map multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data; Edge analysis device 20 is also used to extract more than one candidate component from the field perception data and determine the real-scene component image based on the correlation between the candidate components. The edge analysis device 20 is also used to obtain design component drawings representing the relationship between design components based on the design benchmark model of the construction site, and to determine the correspondence between the actual component drawings and the design component drawings. The result interaction terminal 30 is used to evaluate the matching confidence based on the component correspondence and determine the output component-level recognition result.

[0087] It should be noted that the multimodal acquisition device can be a multi-sensor acquisition device installed on a drone flight platform to acquire various types of sensor data from the construction site. The edge analysis device can be an industrial computing device in a rack-mount or portable case, equipped with an industrial processor, memory, and multiple input interfaces to perform data processing and analysis tasks. The result interaction terminal can be a laptop, industrial tablet, construction dashboard terminal, or cloud management interface to display and output the recognition results.

[0088] It is understood that the multimodal acquisition device, edge analysis device, and result interaction terminal can be connected via wired or wireless communication to form a closed-loop link for on-site identification and verification. The data uploaded by the multimodal acquisition device to the edge analysis device may include image data, thermal imaging data, positioning data, attitude data, and sampling timestamps. The data sent by the edge analysis device to the multimodal acquisition device may include inspection task parameters, re-shooting area parameters, flight altitude parameters, shooting angle parameters, and overlap rate parameters. The data output by the edge analysis device to the result interaction terminal may include component number correspondence, matching confidence, a list of missing components, positional deviation, attitude deviation, and thermal feature binding results. The data sent by the result interaction terminal to the edge analysis device may include verification query requests, area filtering requests, and result export requests.

[0089] Furthermore, referring to Figure 7 , Figure 7This is a schematic diagram of the second structure of the photovoltaic construction component identification system of this application. The multimodal data includes: image data, thermal imaging data, positioning data, and attitude data. The multimodal acquisition device includes: a flight platform. The flight platform is internally equipped with: a positioning unit 101, an image acquisition unit 102, a thermal imaging acquisition unit 103, an attitude acquisition unit 104, a synchronization triggering device 105, and a communication transceiver unit 106. The synchronization triggering device 105 is connected to the first end of the positioning unit 101, the image acquisition unit 102, the thermal imaging acquisition unit 103, and the attitude acquisition unit 104, respectively. The communication transceiver unit 106 is connected to the second end of the positioning unit 101, the image acquisition unit 102, the thermal imaging acquisition unit 103, and the attitude acquisition unit 104, respectively.

[0090] It should be noted that the synchronization triggering device can be used to transmit synchronization triggering signals to the positioning unit, image acquisition unit, thermal imaging acquisition unit, and attitude acquisition unit respectively, in order to establish a unified time reference. The communication transceiver unit can be used to send the positioning data acquired by the image data acquisition unit, the image data acquired by the image acquisition unit, the thermal imaging data acquired by the thermal imaging acquisition unit, and the attitude data acquired by the attitude acquisition unit to the edge analysis device.

[0091] Understandably, the flight platform can be the UAV flight platform itself, used to carry various acquisition units and power supply systems, and to provide flight control and route execution capabilities. The positioning unit can be an RTK positioning unit, used to output positioning data from the construction site. The image acquisition unit can be a visible light imaging unit, used to acquire visible light image data. The thermal imaging acquisition unit can be an infrared thermal imaging unit, used to acquire temperature distribution image data. The attitude acquisition unit can be a flight control attitude acquisition unit or an inertial measurement unit, used to acquire attitude data from the flight platform. The synchronization triggering device can be a synchronization trigger board or an equivalent synchronization circuit, used to generate synchronization trigger signals. The communication transceiver unit can be a wireless communication module, used for data transmission and reception with the edge analysis device. The synchronization trigger signal can be an exposure trigger signal, a timestamp alignment signal, or a sampling synchronization marker.

[0092] Furthermore, the multimodal acquisition device also includes a local buffer unit. The second ends of the positioning unit, image acquisition unit, thermal imaging acquisition unit, and attitude acquisition unit are respectively connected to the communication transceiver unit through the local buffer unit.

[0093] It should be understood that the local cache unit can be a data storage device located inside the flight platform, used for short-term storage of data collected by each acquisition unit. The local cache unit can be a storage medium such as solid-state storage, flash memory chip, or memory card.

[0094] The photovoltaic construction component identification system provided in this application, employing the photovoltaic construction component identification method described in the above embodiments, can solve the technical problem of low identification reliability in large-scale photovoltaic power plant component scenarios. Compared with the prior art, the beneficial effects of the photovoltaic construction component identification system provided in this application are the same as those of the photovoltaic construction component identification method provided in the above embodiments, and other technical features of the photovoltaic construction component identification system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0095] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for identifying photovoltaic construction components, characterized in that, The photovoltaic construction component identification method includes the following steps: Acquire multimodal data of the construction site under a unified time reference; The multimodal data is mapped to the same coordinate system to obtain spatiotemporally synchronized field perception data; Extract more than one candidate component from the site perception data, and determine the real-scene component image based on the correlation between the candidate components; Based on the design benchmark model of the construction site, obtain the design component diagram representing the relationship between the design components, and determine the component correspondence between the real-scene component diagram and the design component diagram; Based on the component correspondence, a matching confidence assessment is performed to determine the output component-level recognition result.

2. The photovoltaic construction component identification method as described in claim 1, characterized in that, The steps for acquiring multimodal data of the construction site under a unified time reference include: Obtain the minimum identification feature size of the component to be identified at the construction site and the number of target pixels covered in the image; The target ground resolution and target flight altitude are determined based on the minimum identification feature size and the number of target pixels covered. Based on the target ground resolution and the target flight altitude, the inspection task parameters are obtained, and based on the inspection task parameters, the multimodal data of each of the components to be identified at the construction site under a unified time reference are obtained.

3. The photovoltaic construction component identification method as described in claim 2, characterized in that, The multimodal data includes image data, thermal imaging data, positioning data, and attitude data. The step of mapping the multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data includes: The pose parameters at each acquisition time are determined based on the positioning data and attitude data. The image data and thermal imaging data are mapped to the same coordinate system based on the pose parameters to obtain spatiotemporally synchronized on-site perception data.

4. The photovoltaic construction component identification method as described in claim 3, characterized in that, The step of extracting more than one candidate component from the site perception data and determining the real-scene component image based on the correlation between the candidate components includes: Based on the component category, candidate components are extracted from the image data and thermal imaging data, and the correlation between the candidate components is determined. Using each of the candidate component objects as nodes, edges are established between the corresponding nodes according to the association between each candidate component object to obtain the real-world component diagram.

5. The photovoltaic construction component identification method as described in claim 1, characterized in that, The step of obtaining design component drawings representing the relationships between design components based on the design benchmark model of the construction site, and determining the component correspondence between the actual component drawings and the design component drawings, includes: Design component drawings representing the relationships between design components are obtained based on the design benchmark model at the construction site. The association between the design component diagram and the actual component diagram is obtained, the topological difference term is calculated, and the matching cost is calculated. A bipartite graph is constructed using the design component and the candidate component as nodes on both sides and the matching cost as edge weights. The correspondence between the components is obtained by solving the bipartite graph.

6. The photovoltaic construction component identification method as described in claim 1, characterized in that, The step of evaluating the matching confidence based on the component correspondence and determining the output component-level recognition result includes: The matching confidence is calculated based on the matching cost, neighborhood consistency, and multimodal consistency corresponding to the component correspondence. When the matching confidence is lower than a preset threshold, the region where the candidate object of the component corresponding to the component is located is determined, and the reshoot task parameters are generated based on the region, the occlusion of the candidate object, and the image clarity. Based on the reshoot task parameters, multimodal data of the target area is reacquired until the matching confidence is not lower than the preset threshold, so as to determine the output component-level recognition result.

7. The photovoltaic construction component identification method as described in claim 6, characterized in that, The step of determining the output component-level recognition result includes: Based on the component correspondence, determine the number correspondence between each design component and the corresponding component candidate object; Based on the number correspondence, the positional and orientation deviations between the design component and the corresponding component candidate object are obtained to determine the output component-level recognition result.

8. A photovoltaic construction component identification system, characterized in that, The photovoltaic construction component identification system includes: A multimodal acquisition device is used to acquire multimodal data from the construction site under a unified time reference. An edge analysis device is used to map the multimodal data to the same coordinate system to obtain spatiotemporally synchronized field perception data; The edge analysis device is also used to extract more than one candidate component from the field perception data, and to determine the real-scene component image based on the correlation between the candidate components. The edge analysis device is also used to obtain design component diagrams representing the relationship between design components based on the design benchmark model of the construction site, and to determine the component correspondence between the real-scene component diagram and the design component diagram; The result interaction terminal is used to evaluate the matching confidence based on the correspondence of the components and determine the output component-level recognition result.

9. The photovoltaic construction component identification system as described in claim 8, characterized in that, The multimodal data includes: image data, thermal imaging data, positioning data, and attitude data; the multimodal acquisition device includes: a flight platform. The flight platform is internally equipped with: a positioning unit, an image acquisition unit, a thermal imaging acquisition unit, an attitude acquisition unit, a synchronization triggering device, and a communication transceiver unit; The synchronization triggering device is connected to the first end of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit, respectively; the communication transceiver unit is connected to the second end of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit, respectively. The synchronization triggering device is used to transmit the synchronization triggering signal to the positioning unit, the image acquisition unit, the thermal imaging acquisition unit and the attitude acquisition unit respectively, so as to establish a unified time reference; The communication transceiver unit is used to send the positioning data acquired by the image data acquisition unit, the image data acquired by the image acquisition unit, the thermal imaging data acquired by the thermal imaging acquisition unit, and the attitude data acquired by the attitude acquisition unit to the edge analysis device.

10. The photovoltaic construction component identification system as described in claim 9, characterized in that, The multimodal acquisition device further includes: a local cache unit; The second ends of the positioning unit, the image acquisition unit, the thermal imaging acquisition unit, and the attitude acquisition unit are respectively connected to the communication transceiver unit through the local cache unit.

Images