Large-space unmanned aerial vehicle automatic inspection method and system based on laser radar

Abstract

The invention discloses a large-space unmanned aerial vehicle automatic inspection method and system based on a laser radar, and belongs to the technical field of computers. An original image acquired by an image acquisition assembly of the unmanned aerial vehicle, an original point cloud acquired by a laser radar, first environment information of an environment where the unmanned aerial vehicle is located, second environment information of an inspection position of the unmanned aerial vehicle and dynamic acquisition parameters are acquired. And determining environment image correction information and environment point cloud correction information based on the first environment information, the second environment information, the dynamic image acquisition parameters and the dynamic point cloud acquisition parameters. And determining an image enhancement parameter and a point cloud enhancement parameter based on the inspection indication information, the inspection position, the environment image correction information and the environment point cloud correction information. And processing the original image and the original point cloud based on the image enhancement parameter and the point cloud enhancement parameter to obtain a target image and a target point cloud. And determining whether the inspection position is abnormal based on the target image and the target point cloud.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for automatic inspection of large - space unmanned aerial vehicles based on lidar. Background Technique

[0002] With the wide application of unmanned aerial vehicles (UAVs) in fields such as agricultural monitoring, disaster assessment, and geographical mapping, using UAVs for shooting can obtain a large number of aerial images, and the use of aerial images can achieve the inspection of specific spaces.

[0003] However, aerial images are greatly affected by factors such as weather. In the face of special weather conditions, the effect of using aerial images to inspect specific spaces is poor, that is, it is impossible to detect abnormalities in specific spaces in a timely manner. Summary of the Invention

[0004] Embodiments of this application provide a method and system for automatic inspection of large - space unmanned aerial vehicles based on lidar, which can improve the effect of using UAVs to inspect specific spaces and detect abnormalities in specific spaces in a timely manner. The technical solutions are as follows: On the one hand, a method for automatic inspection of large - space unmanned aerial vehicles based on lidar is provided. The method includes: Obtain the original image collected by the image acquisition component of the UAV, the original point cloud collected by the lidar, the first environmental information of the environment where the UAV is located, the second environmental information of the inspection position of the UAV, and dynamic acquisition parameters. The dynamic acquisition parameters include inspection indication information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters; Based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters, determine environmental image correction information and environmental point cloud correction information; Based on the inspection indication information, the inspection position, the environmental image correction information, and the environmental point cloud correction information, determine image enhancement parameters and point cloud enhancement parameters; Based on the image enhancement parameters and the point cloud enhancement parameters, process the original image and the original point cloud respectively to obtain a target image and a target point cloud; Based on the target image and the target point cloud, determine whether there is an abnormality at the inspection position.

[0005] On the one hand, a system for automatic inspection of large - space unmanned aerial vehicles based on lidar is provided. The device includes: An acquisition module, configured to acquire the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection position of the drone, and dynamic acquisition parameters, where the dynamic acquisition parameters include inspection indication information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters; A correction information determination module, configured to determine environmental image correction information and environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters; An enhancement parameter determination module, configured to determine image enhancement parameters and point cloud enhancement parameters based on the inspection indication information, the inspection position, the environmental image correction information, and the environmental point cloud correction information; A processing module, configured to process the original image and the original point cloud respectively based on the image enhancement parameters and the point cloud enhancement parameters to obtain a target image and a target point cloud; An anomaly recognition module, configured to determine whether there is an anomaly at the inspection position based on the target image and the target point cloud.

[0006] On the one hand, a computer device is provided, where the computer device includes one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the method for automatic inspection of large-space drones based on lidar.

[0007] On the one hand, a computer-readable storage medium is provided, where at least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the method for automatic inspection of large-space drones based on lidar.

[0008] On the one hand, a computer program product or a computer program is provided, where the computer program product or the computer program includes program code, the program code is stored in a computer-readable storage medium, a processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code so that the computer device executes the above-mentioned method for automatic inspection of large-space drones based on lidar.

[0009] Through the technical solution provided by the embodiments of the present application, the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection position of the drone, and the dynamic acquisition parameters are obtained. Based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters, the environmental image correction information and the environmental point cloud correction information are determined. Based on the inspection instruction information, the inspection position, the environmental image correction information, and the environmental point cloud correction information, the image enhancement parameters and the point cloud enhancement parameters are determined. Based on the image enhancement parameters and the point cloud enhancement parameters, the original image and the original point cloud are processed respectively to obtain the target image and the target point cloud. Based on the target image and the target point cloud, it is determined whether there is an abnormality at the inspection position. The inspection combines multi-dimensional information, and the inspection effect is higher, and the abnormality existing in a specific space can be detected in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0011] Figure 1 is a schematic diagram of the implementation environment of a large-space drone automatic inspection method based on lidar provided by the embodiments of the present application; Figure 2 is a flowchart of a large-space drone automatic inspection method based on lidar provided by the embodiments of the present application; Figure 3 is another flowchart of a large-space drone automatic inspection method based on lidar provided by the embodiments of the present application; Figure 4 is a schematic diagram of the structure of a large-space drone automatic inspection system based on lidar provided by the embodiments of the present application; Figure 5 is a schematic diagram of the structure of a server provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0013] In the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions. It should be understood that there is no logical or chronological dependence between "first", "second", and "nth", nor are the quantity and execution order limited.

[0014] Drone: The abbreviation of Unmanned Aerial Vehicle (UAV), which is an unpiloted aircraft controlled by radio remote control equipment and self - contained program control devices.

[0015] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain better results.

[0016] Machine Learning (ML) is an interdisciplinary field that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize existing knowledge sub - models to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning from demonstration.

[0017] Normalization: Maps a sequence of numbers with different value ranges to the interval (0, 1) for easier data processing. In some cases, the normalized values can be directly implemented as probabilities.

[0018] Embedded Coding: Mathematically represents a correspondence relationship, that is, maps the data in the X space to the Y space through a function F, where the function F is an injective function, and the result of the mapping is structure - preserving. The injective function means that the data after mapping corresponds uniquely to the data before mapping, and structure - preserving means that the size relationship of the data before mapping is the same as that of the data after mapping. For example, if there are data X1 and X2 before mapping, and Y1 and Y2 corresponding to X1 and X2 after mapping. If the data X1 > X2 before mapping, then correspondingly, the data Y1 is greater than Y2 after mapping. For words, it is to map the words to another space for subsequent machine learning and processing.

[0019] Attention weight: Can represent the importance of a certain data during the training or prediction process. Importance indicates the degree of influence of the input data on the output data. Data with high importance has a higher corresponding attention weight value, and data with low importance has a lower corresponding attention weight value. In different scenarios, the importance of data is not the same, and the process of training the attention weight of the model is also the process of determining the importance of data.

[0020] It should be noted that the information involved in this application (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. In addition, the shooting positions and inspection objects of the drones in the embodiments of this application are all shooting positions and inspection objects where the use of drones for shooting is permitted by relevant laws and regulations.

[0021] Figure 1 FIG. is a schematic diagram of an implementation environment of an image data analysis method based on an AI model provided by an embodiment of this application. Refer to Figure 1 , and this implementation environment may include a drone 110, a ground information collection device 120, and a server 140.

[0022] The drone 110 is connected to the server 140 through a wireless network and can send the collected information to the server 140. The drone 110 has an image collection component and an environmental information collection component. Through the image collection component, image collection can be performed, and through the environmental information collection component, environmental information can be collected. In the embodiments of this application, the image collection parameters when the image collection component collects images are dynamic image collection parameters, so as to adapt to more diverse shooting scenarios.

[0023] The ground information collection device 120 is set on the ground within the shooting area of the drone 110. This shooting area is also the area where the drone 110 is used for shooting. There are multiple ground information collection devices 120 in the shooting area, and different ground information collection devices 120 can collect relevant information at different shooting positions. The ground information collection device 120 is connected to the server 140 through a wireless network and can send the collected information to the server 140.

[0024] The server 140 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The server 140 can provide background services for the application programs running on the drone 110 and the ground information collection device 120, so as to execute the steps provided by the embodiments of this application to achieve the purpose of the technical solutions provided by the embodiments of this application.

[0025] After introducing the implementation environment of the embodiments of the present application, the application scenarios of the technical solutions provided by the embodiments of the present application will be described below. The technical solutions provided by the embodiments of the present application are applied in the scenario of analyzing the images collected by the unmanned aerial vehicle (UAV), and the implementation of the technical solutions provided by the embodiments of the present application requires the cooperation of the above-mentioned ground information collection device 120. The ground information collection device 120 can be a fixed collection device or a collection device that can be disassembled and moved. For example, for a specific space, multiple ground information collection devices can be arranged in advance in the specific space, and then the UAV is used to collect images, point cloud data, and environmental information, and the information collected by the UAV and the information collected by the ground information collection device 120 are used to implement the inspection of the specific space, thereby improving the inspection effect.

[0026] After introducing the implementation environment and application scenarios of the present application, the technical solutions provided by the embodiments of the present application will be introduced below. Refer to Figure 2 , taking the server as the execution subject as an example, the method includes the following steps.

[0027] 201. The server obtains the original image collected by the image acquisition component of the UAV, the original point cloud collected by the lidar, the first environmental information of the environment where the UAV is located, the second environmental information of the inspection position of the UAV, and the dynamic acquisition parameters, where the dynamic acquisition parameters include inspection indication information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters.

[0028] Among them, the drone is a drone used for image shooting in a specific space, and the specific space is a space where the use of drones for shooting is permitted by relevant laws and regulations. For example, the specific space is a farmland space, a forest farm space, a park space, a nature reserve, etc. Using the drone can achieve efficient inspection of the specific space, and the specific space is also a large space. The original image is an image directly captured by the drone, that is, an aerial image. In the actual work process, the number of original images captured by the drone is relatively large. For the sake of convenience of description, in the embodiments of the present application, it is described with the number of original images being one, and the processing methods of other original images all belong to the same inventive concept. The original point cloud is the point cloud data directly collected by the drone through the lidar, and the original point cloud is collected at the same time as the original image. The first environmental information is the environmental information of the environment where the drone is located, that is, the environmental information of the position where the drone captures the original image. Generally speaking, the first environmental information is the environmental information in the air. The inspection position is the shooting position corresponding to the original image, and the inspection position is a ground position. The second environmental information is the environmental information collected by the ground information collection device, and the ground information collection device is the ground information collection device closest to the shooting position in the specific space. The second environmental information is the environmental information on the ground when the drone captures the shooting position. The inspection instruction information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters in the dynamic acquisition parameters are all dynamically changeable. The inspection instruction information includes the inspection target and the inspection strategy. The inspection target is the purpose of using the drone to inspect the specific space. For example, when the inspection area is a farmland area, the inspection target can be to determine whether there are pests in the farmland area; when the inspection area is a forest farm area, the inspection target can be to determine whether there are fire hazards in the forest farm. The inspection target is configured by technicians according to the actual situation. In the embodiments of the present application, the inspection instruction information is in the form of natural language. The dynamic image acquisition parameters are determined based on the object attributes of the inspection object, the initial image acquisition parameters of the image acquisition component of the drone, and the flight parameters of the drone. The object attributes of the inspection object are used to represent the object characteristics of the inspection object. The inspection object is an object existing at the inspection position. For example, the inspection object is a crop or an animal at the inspection position, and the object attributes include object type, object size, and object surface material, etc. The initial image acquisition parameters are the image acquisition parameters directly determined by the image acquisition component, and are not the image acquisition parameters finally used to capture the original image. The dynamic image acquisition parameters are the image acquisition parameters finally used to capture the original image. The flight parameters of the drone are used to represent the flight situation of the drone. For example, the flight parameters include flight altitude and flight speed, etc. The dynamic point cloud acquisition parameters are determined based on the object attributes of the inspection object, the initial point cloud acquisition parameters of the lidar of the drone, and the flight parameters of the drone.The initial point cloud acquisition parameters of the lidar of the UAV are the point cloud acquisition parameters defaultly adopted by the lidar, rather than the point cloud acquisition parameters finally used for acquiring the original point cloud. The dynamic point cloud acquisition parameters are the point cloud acquisition parameters finally used for acquiring the original point cloud. The dynamic image acquisition parameters are determined by combining the object attributes of the inspection object in the inspection position, the initial image acquisition parameters of the image acquisition component, and the flight parameters of the UAV, that is, the comprehensive parameters determined by combining the actual states of the inspection object, the image acquisition component, and the UAV. The dynamic image acquisition parameters are very important parameters in the technical solution provided in the embodiments of the present application and are an important part of the inventive concept of the technical solution provided in the embodiments of the present application. Correspondingly, the dynamic point cloud acquisition parameters are determined by combining the object attributes of the inspection object in the inspection position, the initial point cloud acquisition parameters of the lidar, and the flight parameters of the UAV, that is, the comprehensive parameters determined by combining the actual states of the inspection object, the lidar, and the UAV. The dynamic point cloud acquisition parameters are very important parameters in the technical solution provided in the embodiments of the present application and are an important part of the inventive concept of the technical solution provided in the embodiments of the present application.

[0029] 202. The server determines environmental image correction information and environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters.

[0030] Among them, the environmental image correction information is the parameter used to correct the original image, and the environmental point cloud correction information is the parameter used to correct the original point cloud. After correction, more accurate image and point cloud data can be obtained.

[0031] 203. The server determines image enhancement parameters and point cloud enhancement parameters based on the inspection instruction information, the inspection position, the environmental image correction information, and the environmental point cloud correction information.

[0032] Among them, the image enhancement parameters are the parameters used to enhance the original image, and the point cloud enhancement parameters are the parameters used to enhance the original point cloud. The image enhancement parameters and the point cloud enhancement parameters are both enhancement parameters obtained by integrating multi-dimensional information.

[0033] 204. The server processes the original image and the original point cloud based on the image enhancement parameters and the point cloud enhancement parameters respectively to obtain a target image and a target point cloud.

[0034] Among them, the target image and the target point cloud are obtained after enhancement. Compared with the original image and the original point cloud, they have more information and higher accuracy.

[0035] 205. The server determines whether there is an abnormality at the inspection position based on the target image and the target point cloud.

[0036] Through the technical solution provided by the embodiments of the present application, the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection position of the drone, and the dynamic acquisition parameters are obtained. Based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters, the environmental image correction information and the environmental point cloud correction information are determined. Based on the inspection instruction information, the inspection position, the environmental image correction information, and the environmental point cloud correction information, the image enhancement parameters and the point cloud enhancement parameters are determined. Based on the image enhancement parameters and the point cloud enhancement parameters, the original image and the original point cloud are processed respectively to obtain the target image and the target point cloud. Based on the target image and the target point cloud, it is determined whether there is an abnormality at the inspection position. During the inspection, multi-dimensional information is combined, and the inspection effect is higher, and the abnormality existing in a specific space can be discovered in time.

[0037] The above steps 201-205 are a brief introduction to the lidar-based large-space drone automatic inspection method provided by the embodiments of the present application. Next, some examples will be combined to more clearly illustrate the lidar-based large-space drone automatic inspection method provided by the embodiments of the present application. See Figure 3 , taking the server as the execution subject as an example, the method includes the following steps.

[0038] 301. The server obtains the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection position of the drone, and the dynamic acquisition parameters. The dynamic acquisition parameters include inspection instruction information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters.

[0039] Among them, the drone is a drone used for image shooting of a specific space, which is a space where the use of drones for shooting is permitted by relevant laws and regulations. For example, the specific space is a farmland space, a forest farm space, a park space, a nature reserve, etc. Using the drone can achieve efficient inspection of the specific space, and the specific space is also a large space. The original image is an image directly captured by the drone, that is, an aerial image. In the actual working process, the number of original images captured by the drone is large. For the convenience of description, in the embodiments of the present application, the number of original images is taken as one for description, and the processing methods of other original images all belong to the same inventive concept. The original point cloud is the point cloud data directly collected by the drone through the lidar, and the collection time of the original point cloud is the same as that of the original image. The first environmental information is the environmental information of the environment where the drone is located, that is, the environmental information of the position where the drone is located when shooting the original image. Generally speaking, the first environmental information is the environmental information in the air. The inspection position is the shooting position corresponding to the original image, and the inspection position is a ground position. The second environmental information is the environmental information collected by the ground information collection device, which is the ground information collection device closest to the shooting position in the specific space. The second environmental information is the environmental information on the ground when the drone shoots the shooting position. The inspection instruction information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters in the dynamic acquisition parameters are all dynamically variable. The inspection instruction information includes the inspection target and the inspection strategy. The inspection target is the purpose of using the drone to inspect the specific space. For example, when the inspection area is a farmland area, the inspection target may be to determine whether there are pests in the farmland area; when the inspection area is a forest farm area, the inspection target may be to determine whether there is a fire hazard in the forest farm. The inspection target is configured by technicians according to the actual situation. In the embodiments of the present application, the inspection instruction information is in the form of natural language. The dynamic image acquisition parameters are determined based on the object attributes of the inspection object, the initial image acquisition parameters of the image acquisition component of the drone, and the flight parameters of the drone. The object attributes of the inspection object are used to represent the object characteristics of the inspection object. The inspection object is an object existing at the inspection position. For example, the inspection object is a crop or an animal at the inspection position. The object attributes include the object type, the object size, and the object surface material, etc. The initial image acquisition parameters are the image acquisition parameters directly determined by the image acquisition component, not the image acquisition parameters finally used to shoot the original image. The dynamic image acquisition parameters are the image acquisition parameters finally used to shoot the original image. The flight parameters of the drone are used to represent the flight situation of the drone. For example, the flight parameters include the flight height and the flight speed, etc. The dynamic point cloud acquisition parameters are determined based on the object attributes of the inspection object, the initial point cloud acquisition parameters of the lidar of the drone, and the flight parameters of the drone.The initial point cloud acquisition parameters of the lidar of the UAV are the point cloud acquisition parameters defaultly adopted by the lidar, rather than the point cloud acquisition parameters finally used for acquiring the original point cloud. The dynamic point cloud acquisition parameters are the point cloud acquisition parameters finally used for acquiring the original point cloud. The dynamic image acquisition parameters are determined by combining the object attributes of the inspection object in the inspection position, the initial image acquisition parameters of the image acquisition component, and the flight parameters of the UAV, that is, the comprehensive parameters determined by combining the actual states of the inspection object, the image acquisition component, and the UAV. The dynamic image acquisition parameters are very important parameters in the technical solution provided in the embodiments of the present application and are an important part of the inventive concept of the technical solution provided in the embodiments of the present application. Correspondingly, the dynamic point cloud acquisition parameters are determined by combining the object attributes of the inspection object in the inspection position, the initial point cloud acquisition parameters of the lidar, and the flight parameters of the UAV, that is, the comprehensive parameters determined by combining the actual states of the inspection object, the lidar, and the UAV. The dynamic point cloud acquisition parameters are very important parameters in the technical solution provided in the embodiments of the present application and are an important part of the inventive concept of the technical solution provided in the embodiments of the present application.

[0040] In some embodiments, the original image is acquired by the image acquisition component of the UAV. The image acquisition component refers to the hardware with image acquisition capabilities, and the image acquisition parameters of the image acquisition component can be adjusted. After the UAV acquires the original image through the image acquisition component, the original image is sent to the server through the network connection with the server, and the server obtains the original image.

[0041] In some embodiments, the original point cloud is acquired by the lidar of the UAV. The lidar refers to the hardware with point cloud acquisition capabilities, and the point cloud acquisition parameters of the lidar can be adjusted. After the UAV acquires the original point cloud through the lidar, the original point cloud is sent to the server through the network connection with the server, and the server obtains the original point cloud.

[0042] In some embodiments, the first environmental information is collected by an environmental sensor on the drone. For example, the first environmental information includes first environmental light parameters, first environmental temperature, first environmental humidity, and weather type. The first environmental light parameters include parameters related to ambient light. For example, the first environmental light parameters include first ambient light intensity, first light color temperature, first light direction, and first light color. The concepts in the first environmental light parameters are all common concepts in the related art. The environmental sensor is actually an ambient light sensor, and the ambient light sensor can also be a sensor in the related art. The embodiments of the present application do not make any limitations in this regard. In the technical solution provided by the embodiments of the present application, the focus is on processing the collected data. As for the manner of obtaining the corresponding data, those skilled in the art can adopt appropriate methods to implement it. After the first environmental information is collected, the first environmental information is sent to the server through the network connection with the server, and the server obtains the first environmental information. The first environmental information and the original image are both collected by the drone, and the collection time of the first environmental information is the same as the collection time of the original image. That is, the first environmental information is the environmental information collected while collecting the original image.

[0043] In some embodiments, the second environmental information is collected by a ground information collection device. For example, the second environmental information includes second environmental light parameters, second environmental temperature, and second environmental humidity. The concepts in the second environmental light parameters are all common concepts in the related art. The ground information collection device includes an environmental sensor, and the second environmental information can be collected through this environmental sensor. The environmental sensor is actually an ambient light sensor, and the ambient light sensor can also be a sensor in the related art. The embodiments of the present application do not make any limitations in this regard. The second environmental information and the first environmental information are environmental information collected by different hardware. In order to maintain the correspondence between the second environmental information and the first environmental information, the timestamp when collecting the environmental information can be used to find the second environmental information corresponding to the first environmental information. For example, the ground information collection device will periodically send the collected environmental information to the server, and the sent environmental information carries the timestamp of the collection time. When the drone sends the first environmental information to the server, it carries the timestamp corresponding to the first environmental information. The server can use the timestamp corresponding to the first environmental information to find the corresponding second environmental information from the environmental information sent by the ground information collection device. The timestamp corresponding to the second environmental information is the same as the first environmental information, or the second environmental information is the second environmental information with the timestamp closest to the first environmental information.

[0044] In the embodiments of the present application, the dynamic image acquisition parameters are determined based on the object attributes of the inspection object, the initial image acquisition parameters of the image acquisition component of the drone, and the flight parameters of the drone. The determination method of the dynamic image acquisition parameters will be described below.

[0045] In some embodiments, the controller of the drone obtains the object attributes of the inspection object from the server. Based on the object attributes and the flight parameters, the controller determines the correction parameters for the image acquisition parameters. The controller corrects the initial image acquisition parameters based on the correction parameters for the image acquisition parameters to obtain the dynamic image acquisition parameters.

[0046] Among them, the drone transmits the initially acquired real-time images to the server. Here, the initially acquired real-time images are acquired by the image acquisition component of the drone based on the initial image acquisition parameters. The server obtains the initial images, performs object detection on the initial images, and obtains the object attributes of the inspection object in the initial images. The server returns the object attributes to the controller of the drone, and the controller of the drone obtains the object attributes of the inspection object. The difference between the initial image and the original image is that the initial image is acquired based on the initial image acquisition parameters, and the original image is acquired based on the dynamic image acquisition parameters. The server can identify the object attributes of the inspection object in the initial image by using object detection and object recognition methods in related technologies. Or, after the server identifies the object type of the inspection object from the initial image, it can query the object attributes of the inspection object in the object attribute database using the object type. The embodiments of the present application do not limit this. The image acquisition parameter correction coefficient is used to correct the initial image acquisition parameters to obtain the dynamic correction coefficient.

[0047] For example, the controller of the drone obtains the object attributes of the inspection object from the server. The controller extracts features from the object attributes and the flight parameters to obtain the object attribute features of the object attributes and the flight parameter features of the flight parameters. The controller fuses the object attribute features and the flight parameter features to obtain the first correction parameter determination features. The controller performs full connection and normalization on the first correction parameter determination features to obtain the correction parameters for the image acquisition parameters. The controller uses the correction parameters for the image acquisition parameters to correct the initial image acquisition parameters to obtain the dynamic image acquisition parameters.

[0048] Among them, the initial image acquisition parameters include the initial aperture size, the initial shutter speed, the initial sensitivity, and the initial white balance. The correction parameters for the image acquisition parameters include the aperture size correction coefficient, the shutter speed correction coefficient, the sensitivity correction coefficient, and the white balance correction coefficient. Using the correction parameters for the image acquisition parameters to correct the initial image acquisition parameters means multiplying the initial aperture size by the aperture size correction coefficient, multiplying the initial shutter speed by the shutter speed correction coefficient, multiplying the initial sensitivity by the sensitivity correction coefficient, and multiplying the initial white balance by the white balance correction coefficient, so as to obtain the dynamic image acquisition parameters.

[0049] In the embodiments of the present application, the dynamic point cloud acquisition parameters are determined based on the object attributes of the inspection object, the initial point cloud acquisition parameters of the lidar of the unmanned aerial vehicle, and the flight parameters of the unmanned aerial vehicle. The determination method of the dynamic point cloud acquisition parameters will be described below.

[0050] In some embodiments, the controller of the unmanned aerial vehicle obtains the object attributes of the inspection object from the server. The controller determines a point cloud acquisition parameter correction parameter based on the object attributes and the flight parameters. The controller corrects the initial point cloud acquisition parameters based on the point cloud acquisition parameter correction parameter to obtain the dynamic point cloud acquisition parameters.

[0051] Among them, the unmanned aerial vehicle transmits the initially acquired point cloud in real time to the server. The initially acquired point cloud in real time here is acquired by the lidar of the unmanned aerial vehicle based on the initial point cloud acquisition parameters. The server obtains the initial point cloud, performs target detection on the initial point cloud, and obtains the object attributes of the inspection object in the initial point cloud. The server returns the object attributes to the controller of the unmanned aerial vehicle, and the controller of the unmanned aerial vehicle obtains the object attributes of the inspection object. The difference between the initial point cloud and the original point cloud is that the initial point cloud is acquired based on the initial point cloud acquisition parameters, and the original point cloud is acquired based on the dynamic point cloud acquisition parameters. The method for the server to identify the object attributes of the inspection object in the initial point cloud can adopt the target detection and target recognition methods in related technologies. Or, after the server determines the object type of the inspection object, it can query the object attributes of the inspection object in the object attribute database using the object type. The embodiments of the present application do not limit this. The point cloud acquisition parameter correction coefficient is used to correct the initial point cloud acquisition parameters, so as to obtain the dynamic correction coefficient.

[0052] For example, the controller of the unmanned aerial vehicle obtains the object attributes of the inspection object from the server. The controller extracts features from the object attributes and the flight parameters to obtain the object attribute features of the object attributes and the flight parameter features of the flight parameters. The controller fuses the object attribute features and the flight parameter features to obtain the second correction parameter determination feature. The controller performs full connection and normalization on the second correction parameter determination feature to obtain the point cloud acquisition parameter correction parameter. The controller uses the point cloud acquisition parameter correction parameter to correct the initial point cloud acquisition parameters to obtain the dynamic point cloud acquisition parameters.

[0053] Among them, the initial point cloud acquisition parameters include an initial field of view angle, an initial angular resolution, an initial sampling rate, and an initial detection wavelength. The point cloud acquisition parameter correction parameters include a field of view angle correction coefficient, an angular resolution correction coefficient, a sampling rate correction coefficient, and a detection wavelength correction coefficient. Using the point cloud acquisition parameter correction parameters to correct the initial point cloud acquisition parameters means multiplying the initial field of view angle by the field of view angle correction coefficient, multiplying the initial angular resolution by the angular resolution correction coefficient, multiplying the initial sampling rate by the sampling rate correction coefficient, and multiplying the initial detection wavelength by the detection wavelength correction coefficient, so as to obtain dynamic point cloud acquisition parameters.

[0054] In some embodiments, the inspection indication information includes an inspection target and an inspection strategy. The inspection indication information is obtained by the server from the target terminal, and the target terminal is the terminal bound to the UAV, that is, the terminal used by the user.

[0055] 302. The server determines environmental image correction information and environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters.

[0056] Among them, the environmental image correction information is a parameter used to correct the original image, and the environmental point cloud correction information is a parameter used to correct the original point cloud. After correction, more accurate image and point cloud data can be obtained. The first environmental information includes first environmental light parameters, first environmental temperature, first environmental humidity, and weather type. The second environmental information includes second environmental light parameters, second environmental temperature, and second environmental humidity.

[0057] In a possible implementation manner, the server determines environmental light correction parameters based on the first environmental light parameters and the second environmental light parameters. The server determines first temperature and humidity correction parameters and second temperature and humidity correction parameters based on the first environmental temperature, the first environmental humidity, the second environmental temperature, the second environmental humidity, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters. The server determines weather type correction parameters based on the weather type. The server determines the environmental image correction information and the environmental point cloud correction information based on the environmental light correction parameters, the first temperature and humidity correction parameters, the second temperature and humidity correction parameters, and the weather type correction parameters.

[0058] Among them, those skilled in the art can understand that the environmental light has a huge impact on image capture. The environmental light correction parameter is a parameter for correcting the image in combination with the environmental light. It is also found in the experiment that the environmental light also has an impact on the captured point cloud, and the change of the environmental light will cause different degrees of noise in the captured point cloud. Those skilled in the art can understand that the environmental temperature and humidity have a certain impact on image capture. In the embodiments of the present application, the temperature and humidity correction parameter is determined in combination with the environmental temperature, environmental humidity and dynamic image capture parameters. The introduction of the dynamic image capture parameter is to jointly amplify the impact of the environmental temperature and humidity on image capture with the environmental temperature and humidity. Those skilled in the art can understand that the environmental humidity has a certain impact on point cloud capture because water vapor in the atmosphere will absorb and scatter the laser, resulting in a shortened laser transmission distance and increased signal attenuation, thus affecting the detection range and accuracy of the lidar. Generally speaking, the higher the environmental humidity, the greater the impact. In addition, during the experiment, it is found that the temperature also has a certain impact on the point cloud captured by the lidar. This impact may be caused by the difference in water vapor density. Therefore, the temperature is also used to correct the point cloud in the embodiments of the present application. Those skilled in the art can understand that the weather type (sunny, rainy, snowy, hazy, foggy, etc.) has a huge impact on image capture and point cloud capture. Therefore, the present application introduces a weather type correction parameter on the basis of the environmental light correction parameter and the temperature and humidity correction parameter to facilitate more comprehensive image correction and point cloud correction.

[0059] To more clearly illustrate the above embodiments, the above embodiments will be described in several parts below.

[0060] In the first part, the server determines the environmental light correction parameter based on the first environmental light parameter and the second environmental light parameter.

[0061] In a possible implementation manner, the first environmental light parameter includes the first environmental light intensity, the first light color temperature, the first light direction and the first light color, and the second environmental light parameter includes the second environmental light intensity, the second light color temperature and the second light color. The server determines the environmental light intensity difference information based on the first environmental light intensity and the second environmental light intensity. The server determines the light color temperature difference information based on the first light color temperature and the second light color temperature. The server determines the intensity color temperature difference information based on the environmental light intensity difference information and the light color temperature difference information. The server determines the light color difference information based on the first light direction, the first light color and the second light color. The server determines the environmental light correction parameter based on the intensity color temperature difference information and the light color difference information.

[0062] For example, the server determines the ambient light intensity difference information based on the first ambient light intensity and the second ambient light intensity. The server determines the light color temperature difference information based on the first light color temperature and the second light color temperature. The server performs weighted fusion on the ambient light intensity difference information and the light color temperature difference information to obtain the intensity color temperature difference information. The server determines the initial color difference information based on the first light color and the second light color. The server determines the direction color correction parameter based on the first light direction. The server multiplies the direction color correction parameter by the initial color difference information to obtain the light color difference information. The server splices the intensity color temperature difference information and the light color difference information to obtain the ambient difference information. The server substitutes the ambient difference information into the first relational data to obtain the ambient light correction parameter.

[0063] Among them, the weights for weighted fusion are set by technicians according to the actual situation, and the embodiments of the present application do not limit this. The first relational data is a relational function that can represent the relationship between the ambient difference information and the ambient light correction information. The first relational data is set by technicians according to the actual situation or obtained by fitting a large number of parameters, and the embodiments of the present application do not limit this. The correspondence between the first light direction and the direction color correction coefficient is set by technicians according to the actual situation. The server queries in the first relational table using the first light direction and can obtain the corresponding direction color correction parameter. The first relational table stores multiple first light directions and the corresponding direction color correction parameters for each first light direction. The first relational table is set by technicians according to the actual situation.

[0064] In some embodiments, the ambient light correction parameter is a set of numbers, and this set of numbers will be transformed into the ambient light correction matrix corresponding to the ambient light correction parameter in the subsequent process. The size of the ambient light correction matrix is the same as the pixel matrix of the original image. For example, taking the size of the pixel matrix of the original image as 2×2 as an example, if the ambient light correction parameter is (1, 2, 3, 4), then converting the ambient light correction parameter (1, 2, 3, 4) into the ambient light correction matrix is .

[0065] Second part: The server determines the first temperature and humidity correction parameter and the second temperature and humidity correction parameter based on the first ambient temperature, the first ambient humidity, the second ambient temperature, the second ambient humidity, the dynamic image acquisition parameter, and the dynamic point cloud acquisition parameter.

[0066] In a possible implementation, the server determines an environmental temperature correction parameter based on the first environmental temperature and the second environmental temperature. The server determines an environmental humidity correction parameter based on the first environmental humidity and the second environmental humidity. The server determines the first temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic image acquisition parameter. The server determines the second temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic point cloud acquisition parameter.

[0067] To illustrate the above implementation more clearly, the above implementation will be described in several parts below.

[0068] A. The server determines an environmental temperature correction parameter based on the first environmental temperature and the second environmental temperature.

[0069] In a possible implementation, the server determines environmental temperature difference information based on the first environmental temperature and the second environmental temperature. The server determines the environmental temperature correction parameter based on the environmental temperature difference information.

[0070] For example, the server determines environmental temperature difference information based on the first environmental temperature and the second environmental temperature. The server uses the environmental temperature difference information to query in the second relationship table to obtain the environmental temperature correction parameter.

[0071] Among them, the second relationship table stores multiple candidate environmental temperature difference information and the corresponding candidate environmental temperature correction parameters for each candidate environmental temperature difference information. By using the environmental temperature difference information to query in the second relationship table, the corresponding environmental temperature correction parameter can be obtained. The second relationship table is set by those skilled in the art according to the actual situation, and the embodiments of the present application do not limit this.

[0072] In some embodiments, the temperature correction parameter is a set of numbers, and this set of numbers will be transformed into a temperature correction matrix corresponding to the temperature correction parameter in the subsequent process. The size of the temperature correction matrix is the same as the pixel matrix of the original image. In addition, the size of the pixel matrix of the original image is the same as the size of the original point cloud.

[0073] B. The server determines an environmental humidity correction parameter based on the first environmental humidity and the second environmental humidity.

[0074] In a possible implementation, the server determines environmental humidity difference information based on the first environmental humidity and the second environmental humidity. The server determines the environmental humidity correction parameter based on the environmental humidity difference information.

[0075] For example, the server determines the environmental humidity difference information based on the first environmental humidity and the second environmental humidity. The server uses the environmental humidity difference information to query in the third relationship table to obtain the environmental humidity correction parameter.

[0076] Among them, the third relationship table stores multiple candidate environmental humidity difference information and the corresponding candidate environmental humidity correction parameters for each candidate environmental humidity difference information. By using the environmental humidity difference information to query in the third relationship table, the corresponding environmental humidity correction parameter can be obtained. The third relationship table is set by technicians according to the actual situation, and the embodiments of the present application do not limit this.

[0077] In some embodiments, the humidity correction parameter is a set of numbers, and this set of numbers will be transformed into a humidity correction matrix corresponding to the humidity correction parameter in the subsequent process. The size of the humidity correction matrix is the same as the pixel matrix of the original image.

[0078] C. The server determines the first temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic image acquisition parameter.

[0079] In a possible implementation manner, the server inputs the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic image acquisition parameter into the first correction parameter determination model. The first correction parameter determination model extracts features from the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic image acquisition parameter to obtain the third correction parameter determination feature. The server uses the first correction parameter determination model to perform multiple rounds of iterative decoding on the third correction parameter determination feature to obtain the first temperature and humidity correction parameter.

[0080] Among them, the first correction parameter determination model includes a first encoder and a first decoder. The first encoder is used for feature extraction, and the first decoder is used for multiple rounds of iterative decoding. In some embodiments, the first encoder is an encoder constructed based on a fully connected network or a convolutional network, and the first decoder is a decoder constructed based on an attention mechanism. The first encoder and the first decoder can adopt the encoders and decoders in related technologies, and the embodiments of the present application do not limit the structures of the first encoder and the first decoder.

[0081] In some embodiments, the first temperature and humidity correction parameter is a set of numbers, and this set of numbers will be transformed into a temperature and humidity correction matrix corresponding to the first temperature and humidity correction parameter in the subsequent process. The size of the first temperature and humidity correction matrix is the same as the pixel matrix of the original image.

[0082] D. The server determines the second temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic point cloud acquisition parameter.

[0083] In a possible implementation, the server inputs the ambient temperature correction parameter, the ambient humidity correction parameter, and the dynamic point cloud acquisition parameter into a second correction parameter determination model. The second correction parameter determination model extracts features from the ambient temperature correction parameter, the ambient humidity correction parameter, and the dynamic point cloud acquisition parameter to obtain a fourth correction parameter determination feature. The server performs multiple rounds of iterative decoding on the fourth correction parameter determination feature through the second correction parameter determination model to obtain the second temperature and humidity correction parameter.

[0084] Among them, the second correction parameter determination model includes a second encoder and a second decoder. The second encoder is used for feature extraction, and the second decoder is used for multiple rounds of iterative decoding. In some embodiments, the second encoder is an encoder constructed based on a fully connected network or a convolutional network, and the second decoder is a decoder constructed based on an attention mechanism. The second encoder and the second decoder can adopt the encoders and decoders in related technologies. The embodiments of the present application do not limit the structures of the second encoder and the second decoder.

[0085] In some embodiments, the second temperature and humidity correction parameter is a set of numbers, and this set of numbers will be transformed into a temperature and humidity correction matrix corresponding to the second temperature and humidity correction parameter in the subsequent process. The size of the second temperature and humidity correction matrix is the same as that of the original point cloud.

[0086] Part Three: The server determines a weather type correction parameter based on the weather type.

[0087] In a possible implementation, the server queries in a fourth relationship table using the weather type to obtain the weather type correction parameter.

[0088] Among them, multiple candidate weather types and candidate weather type correction parameters corresponding to each candidate weather type are stored in the fourth relationship table. The fourth relationship table is set by technicians according to actual situations, and the embodiments of the present application do not limit this.

[0089] In some embodiments, the weather type correction parameter is a set of numbers, and this set of numbers will be transformed into a weather type correction matrix corresponding to the weather type correction parameter in the subsequent process. The size of the weather type correction matrix is the same as that of the pixel matrix of the original image and the original point cloud.

[0090] Part Four: The server determines the environmental image correction information and the environmental point cloud correction information based on the environmental light correction parameter, the first temperature and humidity correction parameter, the second temperature and humidity correction parameter, and the weather type correction parameter.

[0091] In a possible implementation, the server performs weighted fusion on the ambient light correction parameter, the first temperature and humidity correction parameter, and the weather type correction parameter to obtain the ambient image correction information. The server performs weighted fusion on the ambient light correction parameter, the second temperature and humidity correction parameter, and the weather type correction parameter to obtain the ambient point cloud correction information.

[0092] For example, the server converts the ambient light correction parameter, the first temperature and humidity correction parameter, and the weather type correction parameter into an ambient light correction matrix, a first temperature and humidity correction matrix, and a weather type correction matrix respectively. The server fuses the ambient light correction matrix, the first temperature and humidity correction matrix, and the weather type correction matrix to obtain an ambient image correction matrix, and the ambient image correction matrix is the ambient image correction information. The server converts the ambient light correction parameter, the second temperature and humidity correction parameter, and the weather type correction parameter into an ambient light correction matrix, a second temperature and humidity correction matrix, and a weather type correction matrix respectively. The server fuses the ambient light correction matrix, the second temperature and humidity correction matrix, and the weather type correction matrix to obtain an ambient point cloud correction matrix, and the ambient point cloud correction matrix is the ambient point cloud correction information.

[0093] Among them, the fusion of the ambient light correction matrix, the first temperature and humidity correction matrix, and the weather type correction matrix mentioned above refers to performing weighted fusion on the ambient light correction matrix, the first temperature and humidity correction matrix, and the weather type correction matrix. Correspondingly, the fusion of the ambient light correction matrix, the second temperature and humidity correction matrix, and the weather type correction matrix mentioned above refers to performing weighted fusion on the ambient light correction matrix, the second temperature and humidity correction matrix, and the weather type correction matrix. The weights of the weighted fusion are set by technicians according to actual situations, and the embodiments of the present application do not limit this.

[0094] 303. The server determines an image enhancement parameter and a point cloud enhancement parameter based on the inspection instruction information, the inspection location, the ambient image correction information, and the ambient point cloud correction information.

[0095] Among them, the image enhancement parameter is a parameter used to perform image enhancement on the original image, and the point cloud enhancement parameter is a parameter used to perform point cloud enhancement on the original point cloud. Both the image enhancement parameter and the point cloud enhancement parameter are enhancement parameters obtained by integrating multi-dimensional information.

[0096] In a possible implementation, the server determines the image style description information based on the inspection target and inspection strategy in the inspection indication information. The server determines the point cloud noise description information based on the location attribute of the inspection location. The server determines the image enhancement parameter based on the image style description information and the environmental image correction information. The server determines the point cloud enhancement parameter based on the environmental point cloud correction information and the point cloud noise description information.

[0097] Among them, both the image style description information and the point cloud noise description information are in the form of natural language, which is convenient for technicians to trace back and perform subsequent processing. The location attribute of the inspection location is used to represent the characteristics of the inspection location. For example, the location attribute includes the location surface characteristic, and the location surface characteristic is used to represent the flatness of the surface of the inspection location. The location surface characteristic will affect the reflection of the laser and bring noise. Therefore, in the embodiments of the present application, it is used to determine the point cloud noise information. In some embodiments, the location surface characteristic is obtained by recognizing the original image.

[0098] In order to illustrate the above implementation more clearly, the above implementation will be described in several parts below.

[0099] Part 1: The server determines the image style description information based on the inspection target and inspection strategy in the inspection indication information.

[0100] In a possible implementation, the server determines the reference image style parameter corresponding to the inspection target based on the target description information of the inspection target. The server determines the policy adjustment parameter based on the inspection mode and inspection method in the inspection strategy. The server determines the image style description information based on the reference image style parameter and the policy adjustment parameter.

[0101] Among them, the image style description information includes a contrast enhancement coefficient, an edge sharpening level, and a color balance parameter. The target description information of the inspection target is obtained after converting the description method of the inspection target. For example, the server encodes and decodes the inspection target based on the attention mechanism, and can convert the inspection target into the target description information of the inspection target. This is because the inspection target is configured by technicians and has various description methods. Converting it into the target description information can unify the description method and facilitate subsequent processing. The reference image style parameter is used to describe the image style, and the reference image style parameter is similar to the filter in the related art. The inspection modes include single inspection and periodic inspection, etc. The single inspection includes single fast inspection and single fine inspection, etc. The requirements for the collected images are different under different inspection modes, and are specifically set by technicians according to the requirements. The embodiments of the present application do not limit this. The inspection methods include global inspection and local inspection, etc., and are specifically set by technicians according to the requirements. The embodiments of the present application do not limit this. In the embodiments of the present application, the style requirements for the collected images are different under the combination of different inspection modes and inspection methods to meet the requirements of different inspection modes and inspection methods. The policy adjustment parameter determined based on the inspection mode and inspection method is used to meet the requirements of different inspection modes and inspection methods in terms of image style.

[0102] For example, the server determines a plurality of style keywords that match the target description information. The plurality of style keywords form the style keyword set, and the style keyword set includes at least two of a contrast level, saturation, sharpening intensity, and hue tendency. The server performs semantic encoding on the plurality of style keywords to obtain the keyword semantic features of each style keyword. The server fully connects and normalizes the keyword semantic features of the plurality of style keywords to obtain visual feature parameters. The server uses the visual feature parameters to query in the style mapping table to obtain the reference image style parameter. The style mapping table is used to represent the corresponding relationship between the visual feature parameters and the reference image style parameter. The server uses the inspection mode and the inspection method to query in the fifth relationship table to obtain the policy adjustment parameter. The server respectively extracts features from the reference image style parameter and the policy adjustment parameter to obtain the style parameter feature of the reference image style parameter and the adjustment parameter feature of the policy adjustment parameter. The server fuses the style parameter feature and the adjustment parameter feature to obtain a style description feature. The server performs multi-round iterative decoding on the style description feature to obtain the image style description information.

[0103] Among them, the fifth relationship table stores multiple candidate inspection modes, multiple candidate inspection methods, and multiple candidate policy adjustment parameters. One candidate policy adjustment parameter corresponds to one candidate inspection mode and one candidate inspection method. The fifth relationship table is set by technicians according to the actual situation, and this application embodiment does not limit it. The multi-round iterative decoding is multi-round iterative decoding based on the attention mechanism.

[0104] Second part: The server determines the point cloud noise description information based on the location attributes of the inspection location.

[0105] In a possible implementation manner, the server extracts features from the location surface characteristics in the location attributes to obtain the surface features of the inspection location. The server performs multi-round iterative decoding on the surface features based on the attention mechanism to obtain the point cloud noise description information.

[0106] Third part: The server determines the image enhancement parameters based on the image style description information and the environmental image correction information.

[0107] In a possible implementation manner, the server generates a dynamic contrast adjustment parameter based on the contrast enhancement coefficient in the image style description information and the environmental light correction parameter in the environmental image correction information. The server optimizes the edge sharpening level in the image style description information based on the first temperature and humidity correction parameter in the environmental image correction information to generate a sharpening intensity parameter resistant to environmental interference. The server generates an environment-adaptive color correction parameter based on the weather type correction parameter in the environmental image correction information and the color balance parameter in the image style description information.

[0108] Among them, the image enhancement parameters include the dynamic contrast adjustment parameter, the sharpening intensity parameter resistant to environmental interference, and the environment-adaptive color correction parameter.

[0109] For example, the server combines the contrast enhancement coefficient in the image style description information and the environmental light correction parameter in the environmental image correction information to obtain the dynamic contrast adjustment parameter. The server determines an edge sharpening level correction parameter based on the first temperature and humidity correction parameter in the environmental image correction information. The server uses the edge sharpening level correction parameter to correct the edge sharpening level in the image style description information to obtain the sharpening intensity parameter resistant to environmental interference. The server extracts features from the weather type correction parameter in the environmental image correction information and the color balance parameter in the image style description information to obtain a color correction feature. The server performs full connection and normalization on the color correction feature to obtain the color correction parameter.

[0110] Among them, the correspondence between the first temperature and humidity correction parameter and the edge sharpness level correction parameter is set by the technical personnel according to the actual situation, and the embodiments of the present application do not limit this. Using the edge sharpness level correction parameter to correct the edge sharpness level in the image style description information means multiplying the edge sharpness level correction parameter by the edge sharpness level and then adding the edge sharpness level.

[0111] Part Four: The server determines the point cloud enhancement parameter based on the environmental point cloud correction information and the point cloud noise description information.

[0112] In a possible implementation manner, the server respectively extracts features from the environmental point cloud correction information and the point cloud noise description information to obtain the environmental point cloud correction feature of the environmental point cloud correction information and the point cloud noise description feature of the point cloud noise description information. The server fuses the environmental point cloud correction feature and the point cloud noise description feature to obtain a point cloud enhancement feature. The server performs multiple rounds of iterative decoding on the point cloud enhancement feature to obtain the point cloud enhancement parameter.

[0113] Among them, the point cloud enhancement parameter includes a dynamic noise suppression parameter and a point cloud density compensation parameter.

[0114] It should be noted that in the embodiments of the present application, the method of encoding based on the attention mechanism and decoding based on the attention mechanism are used multiple times. The process of encoding based on the attention mechanism can be implemented by an encoder, and the encoder can be an encoder based on the attention mechanism in the related art, such as the encoder of the BERT network. Correspondingly, the process of decoding based on the attention mechanism can be implemented by a decoder, and the decoder can be a decoder based on the attention mechanism in the related art, such as the decoder of the BERT network. Of course, in addition to BERT, it can also be the encoders and decoders in other models based on the attention mechanism. The embodiments of the present application do not limit this. In addition, the full connection in the embodiments of the present application means multiplying the parameter or feature by the full connection matrix, and normalization means substituting the parameter or feature into the normalization function and processing it by the normalization function. The full connection matrix is obtained through training, and the normalization function is selected by the technical personnel according to the actual situation. For example, the SoftMax function or the ReLu function is selected as the normalization function, etc. The embodiments of the present application do not limit this.

[0115] 304. The server processes the original image and the original point cloud respectively based on the image enhancement parameter and the point cloud enhancement parameter to obtain a target image and a target point cloud.

[0116] Among them, the target image and the target point cloud are obtained after enhancement. Compared with the original image and the original point cloud, they have more information and higher accuracy.

[0117] In a possible implementation, the server performs contrast enhancement, edge sharpening, and color correction on the original image based on the dynamic contrast adjustment parameter, the sharpening intensity parameter for anti-environmental interference, and the color correction parameter for environment adaption in the image enhancement parameters, to generate an intermediate optimized image. The server performs denoising filtering and density compensation on the original point cloud based on the dynamic noise suppression parameter and the point cloud density compensation parameter in the point cloud enhancement parameters, to generate an intermediate optimized point cloud. The server registers and aligns the intermediate optimized image and the intermediate optimized point cloud to obtain a target image and a target point cloud with spatio-temporal consistency.

[0118] Among them, the point cloud enhancement parameters include a dynamic noise suppression parameter generated based on the environmental point cloud correction information and a density compensation parameter generated based on the position surface characteristics of the inspection location. The process of generating the dynamic noise suppression parameter based on the environmental point cloud correction information is to perform feature extraction, fully connected, and normalization on the environmental point cloud correction information. The process of generating the density compensation parameter based on the position surface characteristics is to perform feature extraction, fully connected, and normalization on the position surface characteristics. Registration and alignment are to find the corresponding relationship between the pixel points in the image and the data points in the point cloud, so as to complete the registration of the image and the point cloud. The specific process involves technical methods such as coordinate transformation and similarity matching between pixel points and data points in the point cloud. The specific implementation process can refer to the processing methods in the related technologies, and the embodiments of this application do not limit this.

[0119] 305. The server determines whether there is an abnormality at the inspection location based on the target image and the target point cloud.

[0120] In a possible implementation, the server performs multi-scale feature extraction on the target image to generate image texture features and shape features. The server performs spatial distribution analysis on the target point cloud to extract point cloud density features and geometric topology features. The server performs multi-modal feature fusion on the image texture features, shape features, point cloud density features, and geometric topology features to obtain abnormality features. The server calculates an abnormality score for the abnormality features through an abnormality determination model. When the score value exceeds the preset determination threshold, it is determined that there is an abnormality at the inspection location. In the case where an abnormality is determined, the server outputs the abnormality type with the highest matching degree in the abnormality type set and the three-dimensional spatial coordinates of the inspection location. The abnormality type set is dynamically adapted to the inspection indication information. When the score value does not exceed the preset determination threshold, it is determined that there is no abnormality at the inspection location.

[0121] Among them, the anomaly determination model is obtained through multiple rounds of training with multiple sample data and the annotation information of each sample data. A sample data includes a sample image and a sample point cloud, and the annotation information is used to indicate whether there is an anomaly and the type of anomaly at the inspection location corresponding to the sample data. During the training process of the anomaly determination model, contrastive learning can be used, that is, multiple sample data are divided into positive sample data and negative sample data. The annotation information corresponding to the positive sample data indicates that there is no anomaly, and the annotation information corresponding to the negative sample data indicates that there is an anomaly. The anomaly type set includes multiple anomaly types, and the corresponding anomaly type set can be found through the inspection indication information, that is, there are multiple candidate anomaly type sets. After obtaining the inspection indication information, the anomaly type set that matches the inspection indication information can be determined from multiple candidate anomaly type sets. The preset determination threshold is set by technicians according to the actual situation, and this application embodiment does not limit it. Additionally, The sample image and sample point cloud used by the anomaly determination model during training are the target sample image and target sample point cloud obtained by processing the original sample image and original point cloud through the above steps 302-304. The original sample image is the image collected by the drone, and the original sample point cloud is the point cloud collected by the drone's lidar. During the experiment, the accuracy of directly performing anomaly detection based on the original sample image and sample point cloud is much lower than the result of performing anomaly detection on the target sample image and target sample point cloud. This is because the target sample image and target sample point cloud can carry more abundant information after being processed through the above steps 302-303. The above training actually refers to the fine-tuning process, that is, in this application embodiment, a pre-trained multimodal model can be used, and then the multimodal model is supervised and fine-tuned using the sample image, sample point cloud, and corresponding annotation information. Finally, the anomaly determination model is obtained. The specific way of supervised fine-tuning is not improved in the technical solution provided by this application embodiment, and the relevant technical ways can be referred to.

[0122] To illustrate the above implementation more clearly, the process of the server calculating the anomaly score for the anomaly feature through the anomaly determination model in the above implementation is described below.

[0123] In some embodiments, the server inputs the anomaly feature into the anomaly determination model, and the score mapping unit of the anomaly determination model performs full connection and normalization on the anomaly feature to obtain a score value. The server performs full connection and normalization on the anomaly feature through the anomaly type classification unit of the anomaly recognition model to obtain a probability set of the anomaly type set. The probability set includes multiple probabilities, and one probability corresponds to one anomaly type in the anomaly type set.

[0124] Among them, the anomaly type with the highest matching degree in the set of anomaly types refers to the anomaly type that matches the inspection location, that is, the anomaly type with the highest probability in this probability set.

[0125] 306. When there is an anomaly at the inspection location, the server sends an anomaly prompt message to the target terminal, and the anomaly prompt message is used to prompt that there is an anomaly at the inspection location.

[0126] Among them, the anomaly prompt message carries the anomaly type of the inspection location and the three-dimensional spatial coordinates of the inspection location.

[0127] All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be elaborated one by one here.

[0128] Through the technical solution provided by the embodiment of the present application, the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection location of the drone, and the dynamic acquisition parameters are obtained. Based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters, the environmental image correction information and the environmental point cloud correction information are determined. Based on the inspection instruction information, the inspection location, the environmental image correction information, and the environmental point cloud correction information, the image enhancement parameters and the point cloud enhancement parameters are determined. Based on the image enhancement parameters and the point cloud enhancement parameters, the original image and the original point cloud are processed respectively to obtain the target image and the target point cloud. Based on the target image and the target point cloud, it is determined whether there is an anomaly at the inspection location. Multiple dimensions of information are combined during the inspection, and the inspection effect is higher, and anomalies existing in a specific space can be discovered in time.

[0129] Figure 4 It is a schematic structural diagram of a large-space drone automatic inspection system based on lidar provided by an embodiment of the present application. Refer to Figure 4 , the system includes: an acquisition module 401, a correction information determination module 402, an enhancement parameter determination module 403, a processing module 404, and an anomaly recognition module 405.

[0130] The acquisition module 401 is used to acquire the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection location of the drone, and the dynamic acquisition parameters, and the dynamic acquisition parameters include inspection instruction information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters.

[0131] The correction information determination module 402 is used to determine the environmental image correction information and the environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters.

[0132] An enhancement parameter determination module 403, configured to determine an image enhancement parameter and a point cloud enhancement parameter based on the inspection indication information, the inspection location, the environmental image correction information, and the environmental point cloud correction information.

[0133] A processing module 404, configured to process the original image and the original point cloud respectively based on the image enhancement parameter and the point cloud enhancement parameter to obtain a target image and a target point cloud.

[0134] An anomaly recognition module 405, configured to determine whether there is an anomaly at the inspection location based on the target image and the target point cloud.

[0135] In a possible implementation manner, the first environmental information includes a first environmental light parameter, a first environmental temperature, a first environmental humidity, and a weather type, and the second environmental information includes a second environmental light parameter, a second environmental temperature, and a second environmental humidity. The correction information determination module 402 is configured to determine an environmental light correction parameter based on the first environmental light parameter and the second environmental light parameter. Determine a first temperature and humidity correction parameter and a second temperature and humidity correction parameter based on the first environmental temperature, the first environmental humidity, the second environmental temperature, the second environmental humidity, the dynamic image acquisition parameter, and the dynamic point cloud acquisition parameter. Determine a weather type correction parameter based on the weather type. Determine the environmental image correction information and the environmental point cloud correction information based on the environmental light correction parameter, the first temperature and humidity correction parameter, the second temperature and humidity correction parameter, and the weather type correction parameter.

[0136] In a possible implementation manner, the first environmental light parameter includes a first environmental light intensity, a first light color temperature, a first light direction, and a first light color, and the second environmental light parameter includes a second environmental light intensity, a second light color temperature, and a second light color. The correction information determination module 402 is configured to determine an environmental light intensity difference information based on the first environmental light intensity and the second environmental light intensity. Determine a light color temperature difference information based on the first light color temperature and the second light color temperature. Determine an intensity and color temperature difference information based on the environmental light intensity difference information and the light color temperature difference information. Determine a light color difference information based on the first light direction, the first light color, and the second light color. Determine the environmental light correction parameter based on the intensity and color temperature difference information and the light color difference information.

[0137] In a possible implementation, the correction information determination module 402 is configured to determine an environmental temperature correction parameter based on the first environmental temperature and the second environmental temperature, determine an environmental humidity correction parameter based on the first environmental humidity and the second environmental humidity, determine the first temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic image acquisition parameter, and determine the second temperature and humidity correction parameter based on the environmental temperature correction parameter, the environmental humidity correction parameter, and the dynamic point cloud acquisition parameter.

[0138] In a possible implementation, the enhancement parameter determination module 403 is configured to determine image style description information based on the inspection target and the inspection strategy in the inspection indication information, determine point cloud noise description information based on the location attribute of the inspection location, determine the image enhancement parameter based on the image style description information and the environmental image correction information, and determine the point cloud enhancement parameter based on the environmental point cloud correction information and the point cloud noise description information.

[0139] In a possible implementation, the enhancement parameter determination module 403 is configured to determine the reference image style parameter corresponding to the inspection target based on the target description information of the inspection target, determine a strategy adjustment parameter based on the inspection mode and the inspection method in the inspection strategy, and determine the image style description information based on the reference image style parameter and the strategy adjustment parameter. The image style description information includes a contrast enhancement coefficient, an edge sharpening level, and a color balance parameter.

[0140] In a possible implementation, the enhancement parameter determination module 403 is configured to generate a dynamic contrast regulation parameter based on the contrast enhancement coefficient in the image style description information and the environmental light correction parameter in the environmental image correction information, optimize the edge sharpening level in the image style description information with the first temperature and humidity correction parameter in the environmental image correction information to generate a sharpening intensity parameter resistant to environmental interference, and generate an environment-adaptive color correction parameter based on the weather type correction parameter in the environmental image correction information and the color balance parameter in the image style description information. The image enhancement parameter includes the dynamic contrast regulation parameter, the sharpening intensity parameter resistant to environmental interference, and the environment-adaptive color correction parameter.

[0141] In a possible implementation, the processing module 404 is configured to perform contrast enhancement, edge sharpening, and color correction on the original image based on the dynamic contrast adjustment parameter, the sharpening intensity parameter for anti-environmental interference, and the environment-adaptive color correction parameter in the image enhancement parameters, so as to generate an intermediate optimized image. Based on the dynamic noise suppression parameter and the point cloud density compensation parameter in the point cloud enhancement parameters, perform denoising filtering and density compensation on the original point cloud to generate an intermediate optimized point cloud. Register and align the intermediate optimized image and the intermediate optimized point cloud to obtain a target image and a target point cloud with spatio-temporal consistency. Wherein, the point cloud enhancement parameters include a dynamic noise suppression parameter generated based on the environmental point cloud correction information and a density compensation parameter generated based on the position surface characteristics of the inspection location.

[0142] In a possible implementation, the anomaly recognition module 405 is configured to perform multi-scale feature extraction on the target image to generate image texture features and shape features. Perform spatial distribution analysis on the target point cloud to extract point cloud density features and geometric topology features. Perform multi-modal feature fusion on the image texture features, shape features, point cloud density features, and geometric topology features to obtain anomaly features. Calculate an anomaly score for the anomaly features through an anomaly determination model. When the score value exceeds a preset determination threshold, it is determined that there is an anomaly at the inspection location. In the case of determining that there is an anomaly, output the anomaly type with the highest matching degree in the anomaly type set and the three-dimensional spatial coordinates of the inspection location, and the anomaly type set is dynamically adapted to the inspection indication information.

[0143] It should be noted that: when the above-mentioned lidar-based large-space UAV automatic inspection system performs inspections, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be assigned to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the above-mentioned lidar-based large-space UAV automatic inspection system provided in the embodiment belongs to the same concept as the embodiment of the lidar-based large-space UAV automatic inspection method. For the specific implementation process, please refer to the method embodiment, which will not be elaborated here.

[0144] Through the technical solution provided by the embodiments of the present application, the original image collected by the image acquisition component of the unmanned aerial vehicle (UAV), the original point cloud collected by the lidar, the first environmental information of the environment where the UAV is located, the second environmental information of the inspection location of the UAV, and the dynamic acquisition parameters are obtained. Based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters, the environmental image correction information and the environmental point cloud correction information are determined. Based on the inspection instruction information, the inspection location, the environmental image correction information, and the environmental point cloud correction information, the image enhancement parameters and the point cloud enhancement parameters are determined. Based on the image enhancement parameters and the point cloud enhancement parameters, the original image and the original point cloud are processed respectively to obtain the target image and the target point cloud. Based on the target image and the target point cloud, it is determined whether there is an abnormality at the inspection location. During the inspection, multi-dimensional information is combined, and the inspection effect is higher, and abnormalities existing in a specific space can be detected in time.

[0145] Figure 5 FIG. 4 is a schematic structural diagram of a server provided by an embodiment of the present application. The server 500 may vary greatly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 501 and one or more memories 502. Among them, at least one computer program is stored in the one or more memories 502, and the at least one computer program is loaded and executed by the one or more processors 501 to implement the methods provided by the above various method embodiments. Of course, the server 500 may also have components such as wired or wireless network interfaces, keyboards, and input / output interfaces for input / output. The server 500 may also include other components for implementing the functions of the device, which will not be elaborated here.

[0146] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program. The above computer program can be executed by a processor to complete the method for automatically inspecting a large-space UAV based on lidar in the above embodiments. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0147] In an exemplary embodiment, a computer program product or a computer program is further provided. The computer program product or the computer program includes program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device executes the above-mentioned method for automatically inspecting large-space unmanned aerial vehicles based on lidar.

[0148] In some embodiments, the computer program involved in the embodiments of the present application may be deployed to be executed on a single computer device, or on multiple computer devices located at one location. Or, it may be executed on multiple computer devices distributed at multiple locations and interconnected through a communication network. The multiple computer devices distributed at multiple locations and interconnected through a communication network may form a blockchain system.

[0149] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc.

[0150] The above are only optional embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An automatic inspection method for large - space unmanned aerial vehicles based on lidar, characterized in that, The method includes: Obtaining the original image collected by the image acquisition component of the unmanned aerial vehicle (UAV), the original point cloud collected by the lidar, the first environmental information of the environment where the UAV is located, the second environmental information of the inspection position of the UAV, and dynamic acquisition parameters, where the dynamic acquisition parameters include inspection indication information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters; Determining environmental image correction information and environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters; Determining image enhancement parameters and point cloud enhancement parameters based on the inspection indication information, the inspection position, the environmental image correction information, and the environmental point cloud correction information; Processing the original image and the original point cloud respectively based on the image enhancement parameters and the point cloud enhancement parameters to obtain a target image and a target point cloud; Determining whether there is an abnormality at the inspection position based on the target image and the target point cloud.

2. The method according to claim 1, characterized in that, The first environmental information includes first environmental light parameters, first environmental temperature, first environmental humidity, and weather type, and the second environmental information includes second environmental light parameters, second environmental temperature, and second environmental humidity. Determining the environmental image correction information and the environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters includes: Determining an environmental light correction parameter based on the first environmental light parameter and the second environmental light parameter; Determining a first temperature and humidity correction parameter and a second temperature and humidity correction parameter based on the first environmental temperature, the first environmental humidity, the second environmental temperature, the second environmental humidity, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters; Determining a weather type correction parameter based on the weather type; Determining the environmental image correction information and the environmental point cloud correction information based on the environmental light correction parameter, the first temperature and humidity correction parameter, the second temperature and humidity correction parameter, and the weather type correction parameter.

3. The method according to claim 2, wherein The first environmental light parameter includes first environmental light intensity, first light color temperature, first light direction, and first light color, and the second environmental light parameter includes second environmental light intensity, second light color temperature, and second light color. Determining the environmental light correction parameter based on the first environmental light parameter and the second environmental light parameter includes: Determining environmental light intensity difference information based on the first environmental light intensity and the second environmental light intensity; Determining light color temperature difference information based on the first light color temperature and the second light color temperature; Determining intensity and color temperature difference information based on the environmental light intensity difference information and the light color temperature difference information; Determining light color difference information based on the first light direction, the first light color, and the second light color; Determining the environmental light correction parameter based on the intensity and color temperature difference information and the light color difference information.

4. The method according to claim 2, characterized in that, Determining a first temperature and humidity correction parameter and a second temperature and humidity correction parameter based on the first ambient temperature, the first ambient humidity, the second ambient temperature, the second ambient humidity, the dynamic image acquisition parameter, and the dynamic point cloud acquisition parameter includes: Determining an ambient temperature correction parameter based on the first ambient temperature and the second ambient temperature; Determining an ambient humidity correction parameter based on the first ambient humidity and the second ambient humidity; Determining the first temperature and humidity correction parameter based on the ambient temperature correction parameter, the ambient humidity correction parameter, and the dynamic image acquisition parameter; Determining the second temperature and humidity correction parameter based on the ambient temperature correction parameter, the ambient humidity correction parameter, and the dynamic point cloud acquisition parameter.

5. The method according to claim 1, wherein Determining an image enhancement parameter and a point cloud enhancement parameter based on the patrol instruction information, the patrol location, the environmental image correction information, and the environmental point cloud correction information includes: Determining image style description information based on the patrol target and the patrol strategy in the patrol instruction information; Determining point cloud noise description information based on the location attribute of the patrol location; Determining the image enhancement parameter based on the image style description information and the environmental image correction information; Determining the point cloud enhancement parameter based on the environmental point cloud correction information and the point cloud noise description information.

6. The method according to claim 5, wherein Determining the image style description information based on the patrol target and the patrol strategy in the patrol instruction information includes: Determining a reference image style parameter corresponding to the patrol target based on the target description information of the patrol target; Determining a strategy adjustment parameter based on the patrol mode and the patrol method in the patrol strategy; Determining the image style description information based on the reference image style parameter and the strategy adjustment parameter; Wherein, the image style description information includes a contrast enhancement coefficient, an edge sharpening level, and a color balance parameter.

7. The method according to claim 5, wherein Determining the image enhancement parameter based on the image style description information and the environmental image correction information includes: Generating a dynamic contrast adjustment parameter based on the contrast enhancement coefficient in the image style description information and the environmental light correction parameter in the environmental image correction information; Optimizing the edge sharpening level in the image style description information with the first temperature and humidity correction parameter in the environmental image correction information to generate a sharpening intensity parameter resistant to environmental interference; Generating an environment-adaptive color correction parameter based on the weather type correction parameter in the environmental image correction information and the color balance parameter in the image style description information; Wherein, the image enhancement parameter includes the dynamic contrast adjustment parameter, the sharpening intensity parameter resistant to environmental interference, and the environment-adaptive color correction parameter.

8. The method according to claim 1, characterized in that, Processing the original image and the original point cloud respectively based on the image enhancement parameter and the point cloud enhancement parameter to obtain a target image and a target point cloud includes: Based on the dynamic contrast adjustment parameter, the sharpening intensity parameter for anti-environmental interference, and the color correction parameter for environment adaption in the image enhancement parameters, perform contrast enhancement, edge sharpening, and color correction on the original image to generate an intermediate optimized image; Based on the dynamic noise suppression parameter and the point cloud density compensation parameter in the point cloud enhancement parameters, perform denoising filtering and density compensation on the original point cloud to generate an intermediate optimized point cloud; Register and align the intermediate optimized image and the intermediate optimized point cloud to obtain a target image and a target point cloud with spatio-temporal consistency; Among them, the point cloud enhancement parameters include a dynamic noise suppression parameter generated based on the environmental point cloud correction information and a density compensation parameter generated based on the position surface characteristics of the inspection position.

9. The method according to claim 1, wherein Determining whether there is an abnormality at the inspection position based on the target image and the target point cloud includes: Perform multi-scale feature extraction on the target image to generate image texture features and shape features; Perform spatial distribution analysis on the target point cloud to extract point cloud density features and geometric topology features; Perform multi-modal feature fusion on the image texture features, shape features, the point cloud density features, and geometric topology features to obtain abnormality features; Calculate an abnormality score for the abnormality features through an abnormality determination model. When the score value exceeds a preset determination threshold, determine that there is an abnormality at the inspection position; In the case of determining an abnormality, output the abnormality type with the highest matching degree in the abnormality type set and the three-dimensional spatial coordinates of the inspection position, and the abnormality type set is dynamically adapted to the inspection indication information.

10. An automatic inspection system for large - space unmanned aerial vehicles based on lidar, characterized in that, The system includes: An acquisition module for acquiring the original image collected by the image acquisition component of the drone, the original point cloud collected by the lidar, the first environmental information of the environment where the drone is located, the second environmental information of the inspection position of the drone, and dynamic acquisition parameters, where the dynamic acquisition parameters include inspection indication information, dynamic image acquisition parameters, and dynamic point cloud acquisition parameters; A correction information determination module for determining environmental image correction information and environmental point cloud correction information based on the first environmental information, the second environmental information, the dynamic image acquisition parameters, and the dynamic point cloud acquisition parameters; An enhancement parameter determination module for determining image enhancement parameters and point cloud enhancement parameters based on the inspection indication information, the inspection position, the environmental image correction information, and the environmental point cloud correction information; A processing module for processing the original image and the original point cloud respectively based on the image enhancement parameters and the point cloud enhancement parameters to obtain a target image and a target point cloud; An abnormality recognition module for determining whether there is an abnormality at the inspection position based on the target image and the target point cloud.

Images