Method and device for determining execution progress, storage medium and electronic device

Abstract

The invention discloses an execution progress determination method and device, a storage medium and an electronic device.The method comprises the steps that an unmanned aerial vehicle is controlled to collect multiple image maps of a target area within a preset time period, and the target area is an area for executing a target event; an orthophoto map is generated according to the plurality of image maps, and the orthophoto map is used for indicating all capital construction structures of the target area; inputting the orthophoto map into a deep learning model, and determining a target position of each capital construction structure corresponding to the orthophoto map and a target number of the capital construction structures contained in the target area according to an output result of the deep learning model; and determining the execution progress of the target event according to the target position and the target number. By means of the method, the problem that comprehensive and real-time monitoring of the installation progress of the whole photovoltaic field area is difficult to achieve through manual supervision in the related technology can be solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a method and device for determining an execution progress, a storage medium, and an electronic device. Background Art

[0002] Regarding the progress of the daily installation of photovoltaic panels and brackets in the photovoltaic field during the infrastructure construction period, supervision and management are currently mainly carried out through manual statistics and reporting. Manual supervision often relies on the personal experience and judgment of the supervisor. Due to the existence of human factors, such as fatigue, negligence or misjudgment, errors are prone to occur during the manual supervision process. Manual supervision can usually only cover limited areas or time periods, making it difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field. Especially in large-scale photovoltaic projects, the limitations of manual supervision are more obvious.

[0003] At present, the progress of photovoltaic panels and brackets is mainly monitored by installing cameras and manual inspections in the site. Cameras are installed in key areas of photovoltaic sites, and the installation of photovoltaic panels and brackets is checked in real time through remote video monitoring systems. The video monitoring system can only record key nodes and abnormal situations in the construction process, and provide some basis for subsequent progress analysis and problem solving, but it cannot accurately grasp the precise progress of the construction. Manual supervision can usually only cover limited areas or time periods, and it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic site. Especially in large-scale photovoltaic projects, the limitations of manual supervision are more obvious.

[0004] There is no effective solution to the problem that it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field through manual supervision in related technologies. Summary of the invention

[0005] The embodiments of the present application relate to a method and device for determining an execution progress, a storage medium and an electronic device. Through the embodiments of the present application, the problem in the related art that it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field through manual supervision can be solved.

[0006] According to an embodiment of the embodiments of the present application, a method for determining an execution progress is provided, comprising: controlling a drone to collect multiple image maps of a target area within a preset time period, wherein the target area is an area where a target event is executed; generating an orthophoto map based on the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area; inputting the orthophoto map into a deep learning model to determine a target position of each infrastructure structure corresponding to the orthophoto map and a target number of infrastructure structures included in the target area according to an output result of the deep learning model; and determining the execution progress of the target event based on the target position and the target number.

[0007] In an exemplary embodiment, an orthophoto image is generated based on the multiple image images, including: determining an exterior orientation element corresponding to each image image; reprojecting each image image based on the exterior orientation element to correct each image image; determining an overlapping area corresponding to multiple corrected image images; and splicing the multiple corrected image images based on the overlapping area to generate the orthophoto image.

[0008] In an exemplary embodiment, each image map is reprojected according to the external orientation elements to correct each image map, including: converting the coordinate system corresponding to each image map from the sensor coordinate system to the ground coordinate system according to the external orientation elements; calculating the pixel point position of each pixel point in each image map in the ground coordinate system; and correcting each image map according to multiple pixel point positions.

[0009] In an exemplary embodiment, the orthophoto image is input into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model, including: inputting the orthophoto image into the deep learning model so that the deep learning model outputs a plurality of first bounding boxes corresponding to the orthophoto image, wherein the first bounding box is used to indicate the infrastructure structure; filtering the plurality of first bounding boxes so that the infrastructure structures corresponding to any two of the filtered plurality of second bounding boxes are inconsistent; determining the target position according to the position of each second bounding box in the orthophoto image, and determining the target number according to the number of the second bounding boxes.

[0010] In an exemplary embodiment, determining the execution progress of the target event according to the target position and the target quantity includes: superimposing the second bounding boxes of the target quantity onto the orthophoto map according to each second bounding box and each infrastructure structure corresponding to each second bounding box to generate a visualization image corresponding to the target event; comparing the visualization image with a preset image to determine the execution progress of the target event, wherein the preset image is a design drawing corresponding to the target event determined according to preset software, and the preset image is used to indicate the execution result of the target event.

[0011] In an exemplary embodiment, after the orthophoto image is input into the deep learning model, the method further includes: determining a connection point image corresponding to each infrastructure structure according to the output result; determining whether the connection corresponding to each infrastructure structure meets a preset connection standard according to the connection point image; and sending an alarm signal when it is determined that the connection corresponding to the first infrastructure structure among the multiple infrastructure structures does not meet the preset connection standard.

[0012] In an exemplary embodiment, after determining the execution progress of the target event according to the target position and the target quantity, the method further includes: predicting a first execution progress of the target event within a preset time period according to the execution progress and the execution time of the target event; determining whether the first execution progress is within a preset execution progress range; and if it is determined that the first execution progress does not meet the preset execution progress range, adjusting the execution plan of the target event so that the first execution progress is within the preset execution progress range.

[0013] According to another embodiment of the embodiments of the present application, a device for determining an execution progress is also provided, including: a control module, used to control a drone to collect multiple image maps of a target area within a preset time period, wherein the target area is an area for executing a target event; a generation module, used to generate an orthophoto map based on the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area; an input module, used to input the orthophoto map into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto map and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model; a determination module, used to determine the execution progress of the target event based on the target position and the target number.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0015] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to perform the method.

[0017] In an embodiment of the present application, the drone is controlled to collect multiple image maps of the target area for executing the target time in a preset time period, and an orthophoto map for indicating all infrastructure structures in the target area is generated based on the multiple image maps; the orthophoto map is input into the deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto map and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model; the execution progress of the target event is determined according to the target position and the target number. In other words, the embodiment of the present application generates an orthophoto map based on multiple image maps of the target area, and then determines the target position and target number of each infrastructure structure in the orthophoto map, and the execution progress of the target event can be determined according to the target position and the target number. According to the embodiment of the present application, the problem that it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field through manual supervision in the related technology can be solved, and then the comprehensive and real-time monitoring of the execution progress of the target event can be achieved by collecting image maps of the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal for a method for determining execution progress according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for determining execution progress according to an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of the composition structure of a photovoltaic panel and bracket installation progress monitoring method based on image deep learning according to an optional embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of a photovoltaic panel and bracket installation progress monitoring system based on image deep learning according to an optional embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a visualized image according to an optional embodiment of the present application;

[0024] Figure 6 is a schematic diagram of comparing a visualized image with a CAD analysis result according to an optional embodiment of the present application;

[0025] Figure 7 It is a structural block diagram of a device for determining execution progress according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices; "plurality" means two or more.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device or cloud platform or independent physical server or software platform, wherein the above software platform is run through one or more servers. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for determining a method for executing progress according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the methods in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned methods are implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for determining execution progress is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flowchart of a method for determining execution progress according to an embodiment of the present application, the process includes the following steps:

[0032] Step S202, controlling the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is to be performed;

[0033] Step S204, generating an orthophoto map according to the plurality of image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area;

[0034] Among them, the above-mentioned infrastructure structure may include: photovoltaic panels and brackets.

[0035] Step S206, inputting the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to the output result of the deep learning model;

[0036] Step S208: determining the execution progress of the target event according to the target position and the target quantity.

[0037] In an embodiment of the present application, the drone is controlled to collect multiple image maps of the target area for executing the target time in a preset time period, and an orthophoto map for indicating all infrastructure structures in the target area is generated based on the multiple image maps; the orthophoto map is input into the deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto map and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model; the execution progress of the target event is determined according to the target position and the target number. In other words, the embodiment of the present application generates an orthophoto map based on multiple image maps of the target area, and then determines the target position and target number of each infrastructure structure in the orthophoto map, and the execution progress of the target event can be determined according to the target position and the target number. According to the embodiment of the present application, the problem that it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field through manual supervision in the related technology can be solved, and then the comprehensive and real-time monitoring of the execution progress of the target event can be achieved by collecting image maps of the target area.

[0038] Optionally, the above-mentioned step S204 generates an orthophoto image based on the multiple image images, including: determining the external orientation elements corresponding to each image image; reprojecting each image image according to the external orientation elements to correct each image image; determining the overlapping areas corresponding to multiple corrected image images; and splicing the multiple corrected image images according to the overlapping areas to generate the orthophoto image.

[0039] Among them, each image map is reprojected according to the external orientation elements to correct each image map, including: converting the coordinate system corresponding to each image map from the sensor coordinate system to the ground coordinate system according to the external orientation elements; calculating the pixel point position of each pixel point in each image map in the ground coordinate system; and correcting each image map according to multiple pixel point positions.

[0040] It is understood that the technical solutions for generating orthophotos include:

[0041] (1) Determine the external orientation elements corresponding to each image: The external orientation elements refer to the position and attitude information of the camera (or image acquisition device, in this application, the image acquisition device of the drone) when taking the image (including: the camera's longitude and latitude, altitude, heading angle, pitch angle and roll angle). This information can be obtained through the Global Positioning System (GPS) and Inertial Measurement Unit (IMU) carried by the drone. The purpose of determining the external orientation elements is for subsequent reprojection and stitching operations to ensure that each image can be accurately located at the corresponding position on the ground.

[0042] (2) Reproject each image according to the exterior orientation elements: Reprojection uses known exterior orientation elements and terrain information to remap the pixels in the original image to an idealized plane (i.e., convert the coordinate system corresponding to each image from the sensor coordinate system to the ground coordinate system). This process adjusts the position of each pixel in the new image according to the terrain, thereby eliminating the projection difference caused by the terrain and ensuring that the image is projected vertically on the ground. The reprojected image can more accurately reflect the actual situation on the ground.

[0043] (3) Determine the overlapping areas corresponding to multiple corrected images: In drone aerial photography, adjacent images will have a certain overlapping area, which is to ensure that the entire photovoltaic field can be continuously and completely covered. By determining the overlapping area of ​​each image, it can be ensured that during the stitching process, the boundaries of adjacent images can be smoothly transitioned to avoid stitching gaps or ghosting.

[0044] (4) stitching the multiple corrected images according to the overlapping area to generate the orthophoto image: stitching is the process of stitching multiple reprojected corrected images into a complete orthophoto image. When stitching, it is necessary to use the information of the overlapping area to ensure that the pixels of adjacent images in the overlapping area are aligned through image matching and correction algorithms. The stitched orthophoto image will present a bird's-eye view of the entire photovoltaic field without distortion caused by terrain undulations, which is very suitable for subsequent image recognition and analysis.

[0045] In short, the process of generating orthophotos is a complex image processing process. It makes full use of the exterior orientation elements and terrain information of drone aerial photography. Through reprojection and stitching technology, multiple original images are converted into an image that can accurately reflect the actual conditions on the ground, providing high-quality input data for subsequent deep learning-based photovoltaic panel and bracket detection.

[0046] Optionally, step S206 inputs the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model, including: inputting the orthophoto image into the deep learning model so that the deep learning model outputs multiple first bounding boxes corresponding to the orthophoto image, wherein the first bounding box is used to indicate the infrastructure structure; filtering the multiple first bounding boxes so that the infrastructure structures corresponding to any two second bounding boxes in the filtered multiple second bounding boxes are inconsistent; determining the target position according to the position of each second bounding box in the orthophoto image, and determining the target number according to the number of the second bounding boxes.

[0047] It is understood that the technical solutions for determining the target position and target number of infrastructure structures in the orthophoto map include:

[0048] (1) Inputting the orthophoto into the deep learning model:

[0049] The orthophotos collected and processed by the drone are used as input and fed into a pre-trained deep learning model. The model is usually based on a convolutional neural network (CNN) or a similar architecture, and is trained with a large amount of labeled data to be able to recognize specific objects (i.e., the infrastructure structures of this application, such as photovoltaic panels and brackets).

[0050] (2) The deep learning model outputs multiple first bounding boxes corresponding to the orthophoto image:

[0051] After receiving the input of the image, the deep learning model will output a series of bounding boxes, namely the first bounding boxes. These bounding boxes cover the areas that the deep learning model believes may contain photovoltaic panels or brackets. Each bounding box contains location information, that is, the coordinates in the image, as well as the model's classification information and confidence score for the object in the bounding box.

[0052] (3) Screening multiple first bounding boxes:

[0053] The first bounding box output may contain overlapping or incorrect labels. Therefore, a post-processing step is required to filter and optimize these bounding boxes. Commonly used filtering techniques include: Non-Maximum Suppression (NMS), which removes highly overlapping and low-scoring bounding boxes, retaining only the second bounding boxes that are most likely to accurately depict a single photovoltaic panel or bracket, ensuring that the infrastructure structure indicated by each second bounding box is unique and clear.

[0054] (4) Determine the target position according to the position of each second bounding box in the orthophoto image, and determine the number of targets according to the number of second bounding boxes:

[0055] After the second bounding boxes are determined, they can be used to accurately locate the target position of each photovoltaic panel or bracket on the orthophoto map. In addition, by calculating the total number of second bounding boxes, the target number of photovoltaic panels and brackets that actually exist in the orthophoto map can be determined.

[0056] The above technical solution provides the precise location coordinates of each infrastructure structure and also gives their actual quantity, which is extremely important for monitoring construction progress, evaluating installation efficiency, and subsequent operation and maintenance work.

[0057] In summary, the embodiment of the present application automatically detects and counts the infrastructure structures in the photovoltaic field through a deep learning model, thereby achieving accurate monitoring of the construction progress, improving supervision efficiency and accuracy, reducing human errors and omissions, and providing a scientific basis for construction management.

[0058] Wherein, determining the execution progress of the target event according to the target position and the target quantity includes: superimposing the second bounding boxes of the target quantity onto the orthophoto map according to each second bounding box and each infrastructure structure corresponding to each second bounding box to generate a visualization image corresponding to the target event; comparing the visualization image with a preset image to determine the execution progress of the target event, wherein the preset image is a design drawing corresponding to the target event determined according to preset software, and the preset image is used to indicate the execution result of the target event.

[0059] It is understandable that the steps of determining the execution progress of the target event (eg, the installation of photovoltaic panels and brackets) by processing the obtained image information include:

[0060] (1) Determination of target location and target quantity:

[0061] The target location refers to the specific location information of photovoltaic panels and brackets in the orthophoto image, which is usually represented by the location of the bounding box output by the deep learning model. Each bounding box corresponds to one or more photovoltaic panels or brackets.

[0062] The number of targets refers to the number of photovoltaic panels and brackets detected in the orthophoto. These data are identified and calculated directly from the image by the deep learning model.

[0063] (2) Generate a visualization image corresponding to the target event:

[0064] After determining the target location (i.e., each second bounding box) and the target quantity, these bounding boxes are superimposed on the orthophoto map to generate a visualization image. This allows the distribution and quantity of photovoltaic panels and brackets in the actual site to be intuitively seen, enhancing the readability and understanding of the information.

[0065] (3) Compare with the preset image to determine the execution progress:

[0066] A preset image is an image generated after the design drawing or computer-aided design (CAD) model is parsed, representing the expected layout and quantity of photovoltaic panels and brackets installed during the project design phase. The preset image is generated by preset software and can accurately reflect the design intent.

[0067] By comparing the visualization image with the preset image, the system can automatically calculate the difference between the actual installation progress and the designed progress. This usually involves matching and comparing the actual installation quantity and location in the visualization image with the designed quantity and location in the preset image to determine whether the current installation progress meets or deviates from the designed progress.

[0068] This comparative analysis can not only determine the overall progress, but also identify which specific areas or structures are lagging or ahead of schedule in installation, providing precise data support for construction management and decision-making, helping to adjust construction plans in a timely manner, optimize resource allocation, and ensure that the project proceeds as planned.

[0069] In summary, by processing the orthophotos collected by drones through deep learning models, the system can automatically identify the installation status of photovoltaic panels and brackets, generate intuitive visual images, and compare them with design drawings, effectively monitor and manage construction progress, and improve construction efficiency and quality control level.

[0070] Optionally, after inputting the orthophoto image into the deep learning model in step S206, the method further includes: determining a connection point image corresponding to each infrastructure structure according to the output result; determining whether the connection corresponding to each infrastructure structure meets a preset connection standard according to the connection point image; and sending an alarm signal when it is determined that the connection corresponding to the first infrastructure structure among the multiple infrastructure structures does not meet the preset connection standard.

[0071] It is understandable that the process of monitoring the installation quality of photovoltaic panels, brackets and other infrastructure structures based on image deep learning technology includes:

[0072] (1) Determine the connection point image corresponding to each infrastructure structure according to the output result:

[0073] The image deep learning processing module analyzes the orthophotos to identify infrastructure structures such as photovoltaic panels and brackets and their location information. Since the correct installation of connection points (such as bolts, clamps, etc.) is crucial during the installation of photovoltaic panels and brackets, it is necessary to further focus on these connection points to ensure that their installation meets the standards. This step is to further locate and extract image information of connection points from the identified photovoltaic panels and brackets in preparation for subsequent quality assessment.

[0074] (2) determining whether the connection corresponding to each infrastructure structure meets a preset connection standard according to the connection point image:

[0075] After obtaining the image information of the connection points, the pre-trained deep learning model is used to analyze the images of these connection points to determine whether the installation of the connection points meets the preset connection standards. This may include but is not limited to whether the bolts are tightened to the appropriate position, whether the clamps are properly fitted, whether the connection points are damaged or missing, etc. By learning a large number of images of connection points that meet the standards and those that do not, the deep learning model can accurately identify the status of the connection points and thus evaluate the installation quality of the infrastructure structure.

[0076] (3) When it is determined that the connection corresponding to the first infrastructure structure among the multiple infrastructure structures does not meet the preset connection standard, sending an alarm signal:

[0077] If the system detects that the connection point of a certain infrastructure structure (for example, the first infrastructure structure) does not meet the preset connection standard, the system will immediately send an alarm signal to notify the construction management personnel. The alarm signal can be in the form of SMS, email notification, etc., to ensure that relevant personnel can receive the information in time and take corresponding measures, such as arranging personnel to conduct on-site inspections and repairs, or adjusting the construction plan to avoid further quality problems.

[0078] Through the above steps, the embodiment of the present application can not only monitor the construction progress, but also monitor and evaluate the construction quality in real time, thereby improving the efficiency and accuracy of construction management and ensuring the construction quality and safety of the photovoltaic power station.

[0079] Optionally, after determining the execution progress of the target event according to the target position and the target quantity in step S208, the method further includes: predicting a first execution progress of the target event within a preset time period according to the execution progress and the execution time of the target event; determining whether the first execution progress is within a preset execution progress range; and if it is determined that the first execution progress does not meet the preset execution progress range, adjusting the execution plan of the target event so that the first execution progress is within the preset execution progress range.

[0080] It is understood that the methods of using data analysis and predictive models to optimize the execution progress of projects or tasks include:

[0081] (1) Predicting a first execution progress of a target event within a preset time period based on the execution progress and the execution time of the target event.

[0082] Execution progress: refers to the current degree of completion of the target event, usually expressed as a percentage, for example: a project is 30% completed.

[0083] Target event execution time: refers to the time actually used to execute the target event, which may include past execution time.

[0084] Preset time period: refers to the time range of a plan or forecast, such as the next week or month.

[0085] First execution progress: By analyzing the current execution progress and past execution time, the expected completion progress of the target event at the end of the preset time period is predicted.

[0086] The above technical solution uses existing data and models to predict the progress of the target event in a certain period of time in the future, which helps advance planning and resource allocation.

[0087] (2) Determine whether the first execution progress meets a preset execution progress range.

[0088] Preset execution progress range: refers to the target progress range set in the project management or scheduling plan, for example: it is expected that the completion rate of the target event should be between 40% and 50% at the end of the preset time period.

[0089] After predicting the first execution progress, it is necessary to compare the first execution progress with the preset execution progress range to determine whether the predicted progress is within a reasonable range. If the predicted progress exceeds this range, it may be necessary to take measures to adjust it.

[0090] When it is determined that the first execution progress does not conform to the preset execution progress range, the execution plan of the target event is adjusted so that the first execution progress conforms to the preset execution progress range.

[0091] (3) Execution plan adjustments: This may include increasing resource inputs, adjusting task priorities, reallocating tasks to different teams or individuals, or adjusting the project timeline.

[0092] If the predicted progress does not match the planned progress, the system or project manager needs to make corresponding adjustments to the execution plan of the target event to ensure that the predetermined progress target can be achieved in the end. This may involve the reallocation of resources, adjustment of the schedule, or optimization of the workflow.

[0093] The entire process emphasizes the use of data and predictive models to dynamically adjust project management to ensure that target events can proceed smoothly according to the predetermined schedule and resource plan. It can improve efficiency, reduce delays, and effectively control project risks.

[0094] In order to better understand the process of the above-mentioned method for determining the execution progress, the above-mentioned method for determining the execution progress is described below in conjunction with an optional embodiment, but is not intended to limit the technical solution of the embodiment of the present application.

[0095] At present, in the infrastructure projects of the energy industry, in order to avoid the waste of resources, more and more attention is paid to the monitoring of the installation progress of photovoltaic panels and brackets. In order to solve the problem that manual supervision is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field. Especially in large-scale photovoltaic projects, manual supervision is limited. Therefore, an optional embodiment of the present application proposes a method for monitoring the installation progress of photovoltaic panels and brackets based on image deep learning. By processing the orthophoto data of the power station collected by the drone, the various infrastructure structures are detected and identified and the number is counted separately, and the number of response structures in the CAD analysis results is compared and analyzed to calculate the construction and installation progress of various infrastructure structures at the current moment. Specifically: Figure 3 is a schematic diagram of the composition structure of a photovoltaic panel and bracket installation progress monitoring method based on image deep learning according to an optional embodiment of the present application, as shown in FIG. Figure 3 As shown, a photovoltaic panel and bracket installation progress monitoring method based on image deep learning provided by an optional embodiment of the present application mainly includes: a power station orthophoto data processing module, an image deep learning processing module, and an implementation and design data comparison module. The power station orthophoto data processing module, the image deep learning processing module, and the implementation and design data comparison module are integrated into technical modules to count the number of photovoltaic panels and brackets installed every day, thereby realizing the installation progress monitoring of your photovoltaic panels and brackets. Specifically:

[0096] (1) Power station orthophoto data processing module:

[0097] It is mainly used to control the drone to fly according to the planned route at the preset time period, for example: two preset time periods before and after construction every day. And control the drone to collect clear and shadow-free image data during the flight (connect multiple images). After confirming that the data is correct, use the GPS data and ground control point information carried by the drone to calculate the exterior orientation elements (such as position, posture) of each image through aerial triangulation technology. Use the overlapping areas of multiple images for stitching to generate an orthophoto map that completely covers the power station area. Convert the coordinate system corresponding to the orthophoto map from the sensor coordinate system to the ground coordinate system to eliminate the projection difference caused by terrain undulations. Finally, output an orthophoto that meets the conditions in clarity, color reproduction, and stitching effect.

[0098] (2) Image deep learning processing module:

[0099] According to the task requirements (for example, detection and counting of photovoltaic panels and brackets), automatically select the appropriate deep learning model architecture (for example, convolutional neural network CNN, region proposal network RPN, etc.). Input the orthophoto map to be identified into the trained deep learning model. The deep learning model will automatically detect photovoltaic panels and brackets (i.e., the infrastructure structure of this application) in the image, and output their location information (for example, bounding box, i.e., target location) and quantity (i.e., target quantity). Display the detection results in a visual way on the original image to intuitively view the distribution of photovoltaic panels and brackets.

[0100] (3) Implementation and design data comparison module:

[0101] The detection results displayed on the original image after visualization are counted separately to form a set of statistical data including the location information and quantity of photovoltaic panels and brackets. The statistical data are then compared with the number of response structures in the CAD analysis results to calculate the construction and installation progress of various infrastructure structures at the current moment.

[0102] Figure 4 is a schematic diagram of a photovoltaic panel and bracket installation progress monitoring system based on image deep learning according to an optional embodiment of the present application, such as Figure 4 As shown:

[0103] Steps 1 to 3 are steps performed by the power station orthophoto data processing module.

[0104] Step 1: Use a drone to mount a high-definition camera to collect data at different times of the day.

[0105] Step 2: Using the GPS data and ground control point information carried by the drone, the exterior orientation elements (position and attitude) of each image are calculated through aerial triangulation technology.

[0106] Step 3: Use the overlapping areas of multiple images to generate an orthophoto map that completely covers the power station area. Convert the image from the sensor coordinate system to the ground coordinate system to eliminate the projection difference caused by the terrain undulation.

[0107] Step 4 to step 6 are the steps performed by the image deep learning processing module.

[0108] Step 4: According to the task requirements (for example, detection and counting of photovoltaic panels and brackets), automatically select the appropriate deep learning model architecture (for example, convolutional neural network CNN, region proposal network RPN, etc.) and import it to generate orthophotos.

[0109] Step 5: Input the orthophoto to be identified into the trained model. The model will automatically detect the photovoltaic panels and brackets in the image and output their location information (e.g., bounding box) and quantity.

[0110] Step 6 Figure 5 is a schematic diagram of a visualized image according to an optional embodiment of the present application, such as Figure 5 As shown, the detection results are displayed on the original image in a visual manner (ie, a visual image) so that the distribution of photovoltaic panels and brackets can be viewed intuitively.

[0111] Step 7 is a step performed by the implementation and design data comparison module.

[0112] Step 7 Figure 6 is a schematic diagram of comparing a visualized image with a CAD analysis result according to an optional embodiment of the present application, such as Figure 6 As shown, the construction and installation progress of various infrastructure structures at the current moment is calculated by comparing the number of response structures in the CAD analysis results.

[0113] According to the optional embodiments of the present application, the efficiency and accuracy of supervision can be improved: through automated detection, the image information of key installation components such as photovoltaic panels and brackets can be automatically identified and extracted without manual intervention, which greatly improves the efficiency of supervision. Through the deep learning algorithm, the model can accurately distinguish targets such as photovoltaic panels and brackets, reducing human errors and omissions. Using the position information and size features in the image, the technology can accurately calculate the installation position and quantity of photovoltaic panels and brackets, providing a reliable basis for real-time monitoring of construction progress; promoting scientific construction management and decision-making: through data analysis of construction progress, the technology can predict the trend of construction progress in the future and provide decision-making support for managers. At the same time, bottlenecks and problems in construction can be discovered in time, providing a basis for construction adjustment and optimization. Based on the precise supervision of construction progress, managers can better grasp the needs of construction resources and personnel, thereby optimizing resource allocation and improving construction efficiency and quality; enhancing construction safety and quality control: automatically detecting safety hazards in the construction area during the monitoring process, such as unstable installation of photovoltaic panels and unreasonable bracket structure, and issuing early warning signals in time to ensure construction safety. Through image recognition and quality assessment of photovoltaic panels, brackets and other installation components, this technology can promptly detect quality problems in construction, such as installation that does not meet specifications and material quality that does not meet standards, providing strong support for construction quality control.

[0114] The optional embodiment of the present application defines a touch screen automatic focusing method based on infrared thermal imaging, which divides the touch screen into multiple areas, analyzes and counts the touch points when the touch screen is operated, and then automatically focuses according to the area with the largest proportion of the statistics. The automatic focus then performs dimensional statistics on the grayscale of all points in this area to drive the electric lens to focus.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0116] The present application also provides a structural block diagram of a device for determining execution progress. Figure 7 is a structural block diagram of a device for determining execution progress according to an embodiment of the present application; Figure 7 As shown, including:

[0117] The control module 72 is used to control the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is performed;

[0118] A generating module 74, configured to generate an orthophoto map according to the plurality of image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area;

[0119] An input module 76, used to input the orthophoto image into a deep learning model, so as to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model;

[0120] The determination module 78 is used to determine the execution progress of the target event according to the target position and the target quantity.

[0121] Through the above-mentioned device, the drone is controlled to collect multiple image maps of the target area for executing the target time in a preset time period, and an orthophoto map for indicating all the infrastructure structures in the target area is generated based on the multiple image maps; the orthophoto map is input into the deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto map and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model; the execution progress of the target event is determined according to the target position and the target number. In other words, the embodiment of the present application generates an orthophoto map based on multiple image maps of the target area, and then determines the target position and the target number of each infrastructure structure in the orthophoto map, and the execution progress of the target event can be determined according to the target position and the target number. According to the embodiment of the present application, the problem that it is difficult to achieve comprehensive and real-time monitoring of the installation progress of the entire photovoltaic field through manual supervision in the related technology can be solved, and then the comprehensive and real-time monitoring of the execution progress of the target event can be achieved by collecting image maps of the target area.

[0122] In an exemplary embodiment, the generation module 74 is also used to determine the external orientation elements corresponding to each image map; reproject each image map according to the external orientation elements to correct each image map; determine the overlapping areas corresponding to multiple corrected image maps; and splice the multiple corrected image maps according to the overlapping areas to generate the orthophoto image map.

[0123] In an exemplary embodiment, the generation module 74 is also used to convert the coordinate system corresponding to each image image from the sensor coordinate system to the ground coordinate system according to the external orientation elements; calculate the pixel point position of each pixel point in each image image in the ground coordinate system; and correct each image image according to multiple pixel point positions.

[0124] In an exemplary embodiment, the input module 76 is also used to input the orthophoto image into the deep learning model so that the deep learning model outputs multiple first bounding boxes corresponding to the orthophoto image, wherein the first bounding box is used to indicate the infrastructure structure; the multiple first bounding boxes are filtered so that the infrastructure structures corresponding to any two second bounding boxes in the filtered multiple second bounding boxes are inconsistent; the target position is determined according to the position of each second bounding box in the orthophoto image, and the target number is determined according to the number of the second bounding boxes.

[0125] In an exemplary embodiment, the determination module 78 is further used to superimpose the target number of second bounding boxes onto the orthophoto map according to each second bounding box and each infrastructure structure corresponding to each second bounding box to generate a visualization image corresponding to the target event; and compare the visualization image with a preset image to determine the execution progress of the target event, wherein the preset image is a design drawing corresponding to the target event determined according to preset software, and the preset image is used to indicate the execution result of the target event.

[0126] In an exemplary embodiment, the input module 76 is further used to determine the connection point image corresponding to each infrastructure structure based on the output result; determine whether the connection corresponding to each infrastructure structure meets the preset connection standard based on the connection point image; and send an alarm signal when it is determined that the connection corresponding to the first infrastructure structure among multiple infrastructure structures does not meet the preset connection standard.

[0127] In an exemplary embodiment, the determination module 78 is also used to predict the first execution progress of the target event within a preset time period based on the execution progress and the execution time of the target event; determine whether the first execution progress is within a preset execution progress range; and if it is determined that the first execution progress does not meet the preset execution progress range, adjust the execution plan of the target event so that the first execution progress meets the preset execution progress range.

[0128] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0129] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0130] S1, controlling the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is to be performed;

[0131] S2, generating an orthophoto map according to the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area;

[0132] S3, inputting the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to an output result of the deep learning model;

[0133] S4: Determine the execution progress of the target event according to the target position and the target quantity.

[0134] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0135] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0136] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0137] S1, controlling the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is to be performed;

[0138] S2, generating an orthophoto map according to the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area;

[0139] S3, inputting the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to an output result of the deep learning model;

[0140] S4: Determine the execution progress of the target event according to the target position and the target quantity.

[0141] An embodiment of the present application further provides a computer program product, including a computer program, which is used by a processor to execute the steps in any of the above method embodiments.

[0142] Optionally, in this embodiment, the computer program product may be executed by a processor to perform the following steps:

[0143] S1, controlling the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is to be performed;

[0144] S2, generating an orthophoto map according to the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area;

[0145] S3, inputting the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to an output result of the deep learning model;

[0146] S4: Determine the execution progress of the target event according to the target position and the target quantity.

[0147] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0148] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0149] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0150] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining execution progress, characterized in that: include: Controlling the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is to be performed; Generating an orthophoto map according to the multiple image maps, wherein the orthophoto map is used to indicate all infrastructure structures in the target area; Inputting the orthophoto image into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to the output result of the deep learning model; The execution progress of the target event is determined according to the target position and the target quantity.

2. The method for determining execution progress according to claim 1, characterized in that: Generating an orthophoto map according to the plurality of image maps comprises: Determine the exterior orientation elements corresponding to each image map; Reprojecting each of the image maps according to the exterior orientation elements to correct each of the image maps; Determine overlapping areas corresponding to a plurality of corrected image maps; The multiple corrected image maps are spliced ​​according to the overlapping area to generate the orthophoto map.

3. The method for determining execution progress according to claim 2, characterized in that: Reprojecting each image map according to the exterior orientation elements to correct each image map includes: Converting the coordinate system corresponding to each image from the sensor coordinate system to the ground coordinate system according to the exterior orientation elements; Calculate the pixel position of each pixel in each image in the ground coordinate system; Each image is corrected according to the positions of multiple pixel points.

4. The method for determining execution progress according to claim 1, characterized in that: The orthophoto image is input into a deep learning model to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures included in the target area according to the output result of the deep learning model, including: Inputting the orthophoto image into the deep learning model so that the deep learning model outputs a plurality of first bounding boxes corresponding to the orthophoto image, wherein the first bounding boxes are used to indicate the infrastructure structure; The plurality of first bounding boxes are screened so that the infrastructure structures corresponding to any two second bounding boxes in the screened plurality of second bounding boxes are inconsistent; The target position is determined according to the position of each second bounding box in the orthophoto image, and the number of targets is determined according to the number of the second bounding boxes.

5. The method for determining execution progress according to claim 4, characterized in that: Determining the execution progress of the target event according to the target position and the target quantity includes: superimposing the target number of second bounding boxes onto the orthophoto map according to each second bounding box and each infrastructure structure corresponding to each second bounding box, so as to generate a visualization image corresponding to the target event; The visualization image is compared with a preset image to determine the execution progress of the target event, wherein the preset image is a design drawing corresponding to the target event determined according to preset software, and the preset image is used to indicate the execution result of the target event.

6. The method for determining execution progress according to claim 1, characterized in that: After inputting the orthophoto into the deep learning model, the method further includes: Determine the connection point image corresponding to each infrastructure structure according to the output result; Determining whether the connection corresponding to each infrastructure structure meets a preset connection standard according to the connection point image; When it is determined that the connection corresponding to the first infrastructure structure among the multiple infrastructure structures does not meet the preset connection standard, an alarm signal is sent.

7. The method for determining execution progress according to claim 1, characterized in that: After determining the execution progress of the target event according to the target position and the target quantity, the method further includes: Predicting a first execution progress of the target event within a preset time period according to the execution progress and the execution time of the target event; Determining whether the first execution progress meets a preset execution progress range; When it is determined that the first execution progress does not conform to the preset execution progress range, the execution plan of the target event is adjusted so that the first execution progress conforms to the preset execution progress range.

8. A device for determining execution progress, characterized in that: include: A control module, used to control the drone to collect multiple images of a target area in a preset time period, wherein the target area is an area where a target event is performed; A generating module, configured to generate an orthophoto map according to the plurality of image maps, wherein the orthophoto map is used to indicate all infrastructure structures of the target area; An input module, used to input the orthophoto image into a deep learning model, so as to determine the target position of each infrastructure structure corresponding to the orthophoto image and the target number of infrastructure structures contained in the target area according to the output result of the deep learning model; A determination module is used to determine the execution progress of the target event according to the target position and the target quantity.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images