Event detection method and device based on unmanned aerial vehicle road inspection and electronic device

Abstract

The application discloses an event detection method and device based on unmanned aerial vehicle road inspection and electronic equipment, and relates to the technical field of data processing. The method comprises the following steps: determining the visible light confidence of the unmanned aerial vehicle based on the sensor coefficient and the light parameter corresponding to the visible light image sensor in the unmanned aerial vehicle; determining the infrared thermal imaging confidence of the unmanned aerial vehicle based on the sensor parameter and the environmental state coefficient corresponding to the infrared thermal imaging sensor in the unmanned aerial vehicle; determining the sensor weight corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively based on the flight parameter, the visible light confidence and the infrared thermal imaging confidence; and controlling the visible light image sensor and the infrared thermal imaging sensor based on the sensor weight, detecting the event type of the abnormal event occurring in the target road, and obtaining the target event type. The application solves the technical problem that the detection result of the road abnormal event is inaccurate in the process of inspecting the road based on the prior art.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to an event detection method, apparatus, and electronic device based on unmanned aerial vehicle (UAV) road inspection. Background Technology

[0002] With the rapid development of intelligent transportation systems, unmanned aerial vehicle (UAV) road inspection technology has been widely used due to its high efficiency and flexibility. However, current technologies often rely on single-type sensors (such as visible light cameras) or simple multi-sensor data fusion methods. For example, some systems use only visible light images for target recognition, but are easily affected by environmental factors such as lighting and weather. Other systems, while incorporating infrared thermal imaging data, lack dynamic evaluation of the sensor's own state and environmental conditions, resulting in fixed or unreasonable fusion weights that are difficult to adapt to complex and changing inspection scenarios.

[0003] In summary, the existing technology has the following technical defects: First, it does not fully consider the impact of environmental factors such as humidity and temperature on sensor performance; second, it lacks real-time evaluation of the basic state of the image sensor (such as resolution and color difference); in addition, when fusing visible light and infrared data, the existing methods often ignore the impact of the UAV's flight status (such as speed, altitude, and pitch angle) on image quality, resulting in inaccurate confidence calculation, which ultimately reduces the recognition accuracy. Consequently, this leads to the technical problem of inaccurate detection results of abnormal road events during road inspection based on the existing technology.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides an event detection method, device, and electronic device based on UAV road inspection, to at least solve the technical problem of inaccurate detection results of abnormal road events during road inspection based on existing technologies.

[0006] According to one aspect of this application, an event detection method based on UAV road inspection is provided, comprising: acquiring environmental state coefficients, illumination parameters, and flight parameters of the target road inspected by the UAV, wherein the environmental state coefficients are used to characterize the temperature and humidity information of the target road; determining the visible light confidence level of the UAV based on the sensor coefficients and illumination parameters corresponding to the visible light image sensor in the UAV, wherein the sensor coefficients are used to characterize the resolution and lens chromatic aberration information of the visible light image sensor; determining the infrared thermal imaging confidence level of the UAV based on the sensor parameters and environmental state coefficients corresponding to the infrared thermal imaging sensor in the UAV, wherein the sensor parameters include at least the resolution, thermal sensitivity, and infrared center wavelength of the infrared thermal imaging sensor; determining the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively based on the flight parameters, the visible light confidence level, and the infrared thermal imaging confidence level; and controlling the visible light image sensor and the infrared thermal imaging sensor based on the sensor weights to detect the event type of abnormal events occurring in the target road, thereby obtaining the target event type.

[0007] Optionally, the environmental state coefficient of the target road inspected by the drone is obtained by: comparing the absolute difference between the ambient humidity and the preset humidity with the preset deviation humidity value to obtain the humidity deviation index; comparing the absolute difference between the ambient temperature and the preset temperature with the preset deviation temperature value to obtain the temperature deviation index; and calculating the environmental state coefficient of the target road based on the humidity deviation index and the temperature deviation index.

[0008] Optionally, before determining the visible light confidence level of the UAV based on the sensor coefficients and illumination parameters of the visible light image sensor in the UAV, the event detection method based on UAV road inspection further includes: normalizing the resolution of the visible light image sensor to obtain the resolution index of the visible light image sensor; comparing the lens chromatic aberration of the visible light image sensor with a preset maximum lens chromatic aberration to obtain the chromatic aberration index; and calculating the sensor coefficients of the visible light image sensor based on the resolution index and chromatic aberration index of the visible light image sensor.

[0009] Optionally, based on the sensor coefficients and illumination parameters corresponding to the visible light image sensor in the UAV, the visible light confidence level of the UAV is determined, including: normalizing the ambient visibility and illumination intensity in the illumination parameters to obtain the ambient visibility index and illumination intensity index; performing a ratio processing on the illumination angle in the illumination parameters with a preset maximum illumination angle to obtain the illumination angle index; and calculating the visible light confidence level of the UAV based on the sensor coefficients, ambient visibility index, illumination intensity index, and illumination angle index corresponding to the visible light image sensor.

[0010] Optionally, based on the sensor parameters and environmental state coefficients corresponding to the infrared thermal imaging sensor in the UAV, the confidence level of the infrared thermal imaging corresponding to the UAV is determined, including: normalizing the resolution in the sensor parameters corresponding to the infrared thermal imaging sensor to obtain the resolution index of the infrared thermal imaging sensor; calculating the thermal sensitivity index based on the thermal sensitivity and the preset minimum thermal sensitivity in the sensor parameters; performing a ratio processing on the absolute difference between the infrared center wavelength and the preset infrared center wavelength in the sensor parameters and the preset deviation from the infrared center wavelength value to obtain the wavelength deviation index; and calculating the confidence level of the infrared thermal imaging corresponding to the UAV based on the environmental state coefficient, the resolution index, the thermal sensitivity index, and the wavelength deviation index of the infrared thermal imaging sensor.

[0011] Optionally, based on flight parameters, visible light confidence, and infrared thermal imaging confidence, the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor are determined, including: the angle difference between the target angle and a preset angle in the flight parameters is compared with a preset deviation angle value to obtain a target angle deviation index, where the target angle is the angle formed by the line connecting the UAV and the associated target of the abnormal event and the horizon; the flight speed in the flight parameters is compared with a preset speed to obtain a speed index; the flight altitude in the flight parameters is compared with a preset altitude to obtain an altitude index; based on the target angle deviation index, speed index, and altitude index, the visible light confidence influence factor and the infrared thermal imaging confidence influence factor are calculated; based on the visible light confidence influence factor and the infrared thermal imaging confidence influence factor, the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor are calculated.

[0012] Optionally, based on sensor weights, a visible light image sensor and an infrared thermal imaging sensor are controlled to detect the event types of abnormal events occurring on the target road, thereby obtaining the target event type. This includes: constructing a preset event type set, wherein the preset event type in the preset event type set is one of the following: spilled material, traffic accident, traffic congestion, or road surface damage; controlling the visible light image sensor to detect the probability that an abnormal event belongs to each preset event type in the preset event type set, thereby obtaining a first probability of occurrence for each preset event type; controlling the infrared thermal imaging sensor to detect the probability that an abnormal event belongs to each preset event type in the preset event type set, thereby obtaining a second probability of occurrence for each preset event type; based on the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, a weighted sum of the first and second probabilities of occurrence for each preset event type is performed to obtain a target probability of occurrence for each preset event type; and selecting the preset event type with the highest target probability of occurrence in the preset event type set as the target event type.

[0013] According to another aspect of this application, an event detection device based on UAV road inspection is also provided, comprising: a parameter acquisition unit, used to acquire environmental state coefficients, illumination parameters, and flight parameters of the UAV for the target road being inspected by the UAV, wherein the environmental state coefficients are used to characterize the temperature and humidity information of the target road; a visible light confidence determination unit, used to determine the visible light confidence of the UAV based on the sensor coefficients and illumination parameters corresponding to the visible light image sensor in the UAV, wherein the sensor coefficients are used to characterize the resolution information and lens chromatic aberration information of the visible light image sensor; an infrared thermal imaging confidence determination unit, used to determine the infrared thermal imaging confidence of the UAV based on the sensor parameters and environmental state coefficients corresponding to the infrared thermal imaging sensor in the UAV, wherein the sensor parameters include at least the resolution, thermal sensitivity, and infrared center wavelength of the infrared thermal imaging sensor; a sensor weight determination unit, used to determine the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively based on the flight parameters, the visible light confidence, and the infrared thermal imaging confidence; and a target detection unit, used to control the visible light image sensor and the infrared thermal imaging sensor based on the sensor weights to detect the event type of abnormal events occurring in the target road and obtain the target event type.

[0014] According to another aspect of this application, a computer program product is also provided, which stores a computer program, wherein, when the computer program is running, it controls the computer program product to execute any of the above-mentioned event detection methods based on UAV road inspection.

[0015] According to another aspect of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the event detection method based on UAV road inspection as described above.

[0016] In this application, the environmental state coefficient, illumination parameters, and flight parameters of the target road inspected by the UAV are first obtained. The environmental state coefficient is used to characterize the temperature and humidity information of the target road. Then, based on the sensor coefficient and illumination parameters of the visible light image sensor in the UAV, the application determines the visible light confidence level of the UAV. The sensor coefficient is used to characterize the resolution and lens chromatic aberration information of the visible light image sensor. Next, based on the sensor parameters and environmental state coefficient of the infrared thermal imaging sensor in the UAV, the application determines the infrared thermal imaging confidence level of the UAV. The sensor parameters include at least the resolution, thermal sensitivity, and infrared center wavelength of the infrared thermal imaging sensor. Subsequently, based on the flight parameters, visible light confidence level, and infrared thermal imaging confidence level, the application determines the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, respectively. Finally, based on the sensor weights, the application controls the visible light image sensor and the infrared thermal imaging sensor to detect the event type of abnormal events occurring in the target road, thereby obtaining the target event type.

[0017] As can be seen from the above, this application quantifies the impact of temperature and humidity information in the road inspection environment on the performance of infrared thermal imaging sensors in UAVs based on environmental state coefficients, thereby obtaining infrared thermal imaging confidence. At the same time, this application quantifies the impact of illumination information in the road inspection environment on the performance of visible light sensors in UAVs based on illumination parameters, thereby obtaining visible light confidence. Thus, the purpose of accurately evaluating visible light confidence and infrared thermal imaging confidence is achieved.

[0018] Subsequently, based on the UAV's flight parameters, visible light confidence level, and infrared thermal imaging confidence level, this application adjusts the sensor weights used by the visible light image sensor and the infrared thermal imaging sensor when fusing detection results. This achieves adaptive fusion of detection data from multiple sensors under different road inspection environments and UAV flight conditions, thereby improving the detection accuracy of abnormal events occurring on inspected roads. It also solves the technical defects of rigid weight allocation and inaccurate weight allocation results caused by the lack of environmental interference quantification and incomplete sensor state evaluation during the fusion of detection data from multiple sensors. Furthermore, it solves the technical problem of inaccurate detection results of abnormal events on roads during road inspections based on existing technologies. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1This is a flowchart of an optional event detection method based on UAV road inspection according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of an optional intelligent identification method for unmanned aerial vehicle (UAV) road inspection based on a big data model, according to an embodiment of this application.

[0022] Figure 3 This is a schematic diagram of an optional event detection device based on unmanned aerial vehicle (UAV) road inspection according to an embodiment of this application;

[0023] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should also be noted that all information and data (including but not limited to information used for display and analysis) involved in this application are authorized by the user or fully authorized by all parties. For example, if there is an interface between this system and the relevant user or organization, before obtaining the relevant information, it is necessary to send a request to the aforementioned user or organization through the interface, and obtain the relevant information only after receiving consent from the aforementioned user or organization.

[0027] Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of relevant information and data involved in this application all comply with the relevant laws, regulations, and standards of the relevant regions, and necessary confidentiality measures have been taken. This application does not violate public order and good morals. In addition, this application provides a corresponding operation entry point for users to choose to agree to or refuse authorization. If the user chooses to refuse authorization, the corresponding expert decision-making process will be initiated.

[0028] The present invention will now be described in detail with reference to various embodiments.

[0029] Example 1

[0030] According to an embodiment of this application, an embodiment of an event detection method based on UAV road inspection is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This application provides an event detection system (hereinafter referred to as the detection system) based on UAV road inspection for executing the event detection method based on UAV road inspection in this application. Figure 1 This is a flowchart of an optional event detection method based on unmanned aerial vehicle (UAV) road inspection according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0032] Step S101: Obtain the environmental state coefficient, illumination parameters, and flight parameters of the target road inspected by the UAV. The environmental state coefficient is used to characterize the temperature and humidity information of the target road.

[0033] Optionally, the environmental state coefficient is used to assess the impact of the current inspection environment on the performance of the sensors carried by the UAV. The calculation process of the environmental state coefficient takes into account the deviation between the humidity and the preset humidity, and the temperature and the preset temperature. The environmental state coefficient is calculated by exponential method to assess the comprehensive impact of changes in road environment humidity and temperature on sensor performance.

[0034] Optionally, illumination parameters refer to the current illumination conditions of the target road environment, including visibility, light intensity, and illumination angle. Illumination parameters can directly affect the imaging quality of visible light image sensors.

[0035] Optionally, the drone's flight parameters, including the drone's flight speed, flight altitude, and target angle (the angle between the line connecting the drone and the target and the ground), help to assess the reliability of sensor data during drone inspections. Among these parameters, excessively high flight speed may cause the detection image to become blurry, changes in flight altitude affect the sensor's coverage and resolution, and the target angle affects the angle of light incidence, which may lead to a decrease in image quality.

[0036] In step S101, the detection system comprehensively collects environmental conditions of the road environment and flight conditions of the UAV, providing a quantitative evaluation basis for subsequent calculation of the confidence levels of the visible light image sensor and the infrared thermal imaging sensor. This enhances the environmental adaptability and robustness of the entire detection system. In particular, under complex weather and flight conditions, it can effectively avoid the external influence on the data source collected by the sensors, thereby improving the accuracy of the final road inspection results.

[0037] Step S102: Based on the sensor coefficients and illumination parameters corresponding to the visible light image sensor in the UAV, determine the visible light confidence level of the UAV. The sensor coefficients are used to characterize the resolution information and lens chromatic aberration information of the visible light image sensor.

[0038] Optionally, the sensor coefficient is used to measure the basic performance of a visible light image sensor, including resolution and lens chromatic aberration. High resolution can capture more image details, while lens chromatic aberration is used to evaluate the sharpness of the image.

[0039] In step S102, the visible light confidence level of the UAV is determined based on the sensor coefficients and illumination parameters of the visible light image sensor in the UAV, and a detection model for visible light confidence level is constructed. This model achieves accurate quantification of the reliability of the detection results of the visible light image sensor. The model not only considers the resolution and color difference of the sensor itself, but also combines the real-time ambient lighting conditions, ensuring that the detection system can accurately evaluate the quality of the visible light image under different lighting levels, thereby giving appropriate weights in the subsequent fusion process and improving the accuracy of abnormal event identification.

[0040] Step S103: Based on the sensor parameters and environmental state coefficients corresponding to the infrared thermal imaging sensor in the UAV, determine the infrared thermal imaging confidence level of the UAV. The sensor parameters include at least the resolution, thermal sensitivity and infrared center wavelength of the infrared thermal imaging sensor.

[0041] Optionally, sensor parameters refer to the performance indicators of the infrared thermal imaging sensor, including resolution, thermal sensitivity, and infrared center wavelength. Thermal sensitivity reflects the sensor's ability to detect minute temperature differences, while the choice of infrared center wavelength affects the image's clarity and contrast.

[0042] In step S103, the infrared thermal imaging confidence level of the UAV is determined based on the sensor parameters and environmental state coefficients of the infrared thermal imaging sensor in the UAV. An infrared thermal imaging confidence level detection model is constructed, which realizes the purpose of quantifying the impact of environmental conditions and sensor performance on infrared image quality. The model considers how the environmental state coefficient changes the thermal noise and detection efficiency of the infrared sensor, and how sensor parameters such as resolution and thermal sensitivity affect the imaging quality. This enables the detection system to accurately identify abnormal road events, such as debris and accident scenes at night or in foggy weather, even under adverse weather conditions.

[0043] Step S104: Based on flight parameters, visible light confidence level, and infrared thermal imaging confidence level, determine the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively.

[0044] In step S104, a dynamic weight allocation mechanism is established to achieve effective fusion of detection data from visible light image sensors and infrared thermal imaging sensors in different scenarios. This mechanism can dynamically adjust the fusion weights of the two sensors based on actual road environment conditions and UAV flight conditions, ensuring that the detection system can prioritize the detection data from the more reliable sensor when the UAV is in different flight states or road environment conditions, thereby improving the accuracy of road anomaly detection.

[0045] Step S105: Based on sensor weights, control the visible light image sensor and the infrared thermal imaging sensor to detect the event type of abnormal events occurring in the target road and obtain the target event type.

[0046] Optionally, the types of abnormal events include: spilled objects, traffic accidents, traffic congestion, and road surface damage.

[0047] In step S105, the detection system, through the detection result fusion model after dynamic weight adjustment, can more accurately identify and classify the event type of abnormal events. This step, based on the previously calculated sensor weights, realizes adaptive fusion detection of visible light and infrared data, enabling the detection system to maintain high-precision event type detection even under extreme environmental conditions or unstable drone flight conditions. For example, using this technical solution, it is possible to automatically reduce the visible light confidence weight in hazy weather, suppress the influence of motion blur on visible light data during high-speed flight, and compensate for imaging quality loss through the basic state coefficient when there is chromatic aberration in the lens, thereby improving the robustness of road abnormal event detection and thus improving the quality and efficiency of road inspection results.

[0048] As can be seen from the above, this application quantifies the impact of temperature and humidity information in the road inspection environment on the performance of infrared thermal imaging sensors in UAVs based on environmental state coefficients, thereby obtaining infrared thermal imaging confidence. At the same time, this application quantifies the impact of illumination information in the road inspection environment on the performance of visible light sensors in UAVs based on illumination parameters, thereby obtaining visible light confidence. Thus, the purpose of accurately evaluating visible light confidence and infrared thermal imaging confidence is achieved.

[0049] Subsequently, based on the UAV's flight parameters, visible light confidence level, and infrared thermal imaging confidence level, this application adjusts the sensor weights used by the visible light image sensor and the infrared thermal imaging sensor when fusing detection results. This achieves adaptive fusion of detection data from multiple sensors under different road inspection environments and UAV flight conditions, thereby improving the detection accuracy of abnormal events occurring on inspected roads. It also solves the technical defects of rigid weight allocation and inaccurate weight allocation results caused by the lack of environmental interference quantification and incomplete sensor state evaluation during the fusion of detection data from multiple sensors. Furthermore, it solves the technical problem of inaccurate detection results of abnormal events on roads during road inspections based on existing technologies.

[0050] In one optional embodiment, in order to obtain the environmental state coefficient of the target road inspected by the drone, the detection system compares the absolute difference between the ambient humidity and the preset humidity with the preset deviation humidity value to obtain a humidity deviation index. Then, the detection system compares the absolute difference between the ambient temperature and the preset temperature with the preset deviation temperature value to obtain a temperature deviation index. Finally, the detection system calculates the environmental state coefficient of the target road based on the humidity deviation index and the temperature deviation index.

[0051] Optionally, after the detection system obtains the temperature deviation index and humidity deviation index, the detection system constructs an environmental state model based on the temperature deviation index and humidity deviation index, and outputs the environmental state coefficient. The environmental state model is shown in the following formula (1):

[0052] (1)

[0053] In the above formula (1), Represents the environmental state coefficient. This indicates the humidity deviation index. This indicates the temperature deviation index. ,and The higher the value, the better the environmental condition of the road.

[0054] Optionally, ambient humidity refers to the water vapor content in the air. Specifically, a humidity sensor can be used to measure ambient humidity data in real time. The impact of humidity on the thermal sensitivity of the infrared thermal imaging sensor can be quantified by calculating the proportion of the absolute difference between the humidity and the preset optimal humidity to the allowable deviation threshold.

[0055] Optionally, ambient temperature refers to the air temperature of the environment in which the drone is located. Specifically, a temperature sensor can be used to collect real-time temperature data, and the degree of interference of temperature change on the thermal noise of the infrared sensor can be characterized by calculating the ratio of the absolute difference between the temperature and the preset optimal temperature to the allowable deviation threshold.

[0056] Optionally, the environmental state coefficient is a quantitative indicator that non-linearly fuses the humidity deviation index and the temperature deviation index using an exponential function, specifically employing an exponential decay function. This is achieved by making the coefficient decrease exponentially as the degree of deviation from humidity or temperature increases, and it is used to dynamically correct the calculation results of the infrared thermal imaging confidence model.

[0057] In the above embodiment, the detection system first acquires the current ambient humidity and temperature values ​​using a humidity sensor and a temperature sensor, respectively. The ambient humidity is compared with a preset optimal humidity value (i.e., preset humidity). The absolute difference between the two is calculated and divided by a threshold that allows deviation from the optimal humidity to obtain a humidity deviation index. For example, when the optimal humidity is set to 50% and the allowable deviation is 20%, if the actual humidity is 65%, the humidity deviation index is |65%-50%| / 20%=0.75.

[0058] Similarly, the detection system compares the ambient temperature with the preset optimal temperature (i.e., the preset temperature), calculates the temperature deviation index, and then substitutes the two indices into the environmental state model. When the degree of deviation from humidity or temperature increases, the exponential term... or Enlargement, leading to The value decreases, and the environmental state coefficient decreases. As a multiplier applied to the infrared thermal imaging confidence model, the confidence weight of the infrared sensor is automatically reduced when environmental conditions deteriorate, thereby offsetting the negative impact of humidity and temperature changes on sensor performance.

[0059] Compared to existing technologies, traditional methods lack a quantitative correlation model between humidity and temperature and the performance of infrared sensors, relying solely on fixed thresholds to determine environmental conditions. This proposed solution, however, dynamically calculates humidity and temperature deviation indices and constructs environmental state coefficients using exponential functions. This accurately reflects the combined effect of humidity and temperature, enabling nonlinear modeling of the impact of environmental factors on sensor performance. For example, in high-temperature and high-humidity environments, existing technologies may only trigger a binary alarm signal, while this solution can quantitatively calculate... This is directly used in the calculation process to adjust the infrared confidence level.

[0060] Through the technical solutions described in the above embodiments, the detection system solves the problem of inaccurate detection confidence of infrared thermal imaging sensors caused by changes in environmental humidity and temperature. By calculating the humidity and temperature deviation index in real time and constructing an exponential environmental state coefficient, the infrared thermal imaging confidence model can dynamically perceive changes in environmental conditions. For example, in heavy rain, the humidity deviation index increases significantly, and the environmental state coefficient decreases accordingly, automatically reducing the weight of infrared sensors affected by humidity and avoiding misjudgments caused by lens condensation or decreased thermal sensitivity. Simultaneously, in scenarios with large day-night temperature differences, the dynamic calculation of the temperature deviation index can effectively suppress the impact of thermal noise fluctuations on infrared image quality, improving the reliability of identifying targets such as road debris and road surface damage in complex environments.

[0061] In one optional embodiment, in order to determine the sensor coefficients corresponding to the visible light image sensor in the UAV, the detection system first normalizes the resolution of the visible light image sensor to obtain the resolution index of the visible light image sensor. Then, the detection system compares the lens chromatic aberration of the visible light image sensor with a preset maximum lens chromatic aberration to obtain the chromatic aberration index. Finally, based on the resolution index and chromatic aberration index of the visible light image sensor, the detection system calculates the sensor coefficients corresponding to the visible light image sensor.

[0062] Optionally, after the detection system obtains the resolution index and the color difference index, the detection system constructs a basic state model of the image sensor based on the color difference index and the light image resolution index, and outputs the sensor coefficients corresponding to the visible light image sensor. The basic state model of the image sensor is shown in the following formula (2):

[0063] (2)

[0064] In the above formula (2), This represents the sensor coefficients corresponding to the visible light image sensor. Indicates the shape adjustment factor. Indicates the lens chromatic aberration index. Indicates the light image resolution index. Represents the weighting coefficient, and , ,and The higher the value, the better the basic performance of the high-resolution visible light image sensor.

[0065] Optionally, the lens chromatic aberration index refers to the ratio of the actual chromatic aberration of the lens to the preset maximum permissible chromatic aberration (i.e., the preset maximum lens chromatic aberration). Specifically, it can be achieved by dividing the chromatic aberration measurement value by the standard permissible value, and is used to quantify the negative impact of chromatic aberration on image quality.

[0066] Optionally, the visible light image resolution index refers to the normalized value of the sensor resolution within a preset range. Specifically, it can be achieved by using a linear transformation of (current resolution - minimum resolution) / (maximum resolution - minimum resolution) to eliminate the difference in dimensions between different resolutions.

[0067] Optionally, the image sensor fundamental state coefficient refers to the evaluation index that integrates the effects of color difference and resolution. Specifically, it can be implemented using a mathematical model that combines hyperbolic tangent function and linear weighted combination. The color difference threshold effect is reflected by a nonlinear function, and the influence weights of color difference and resolution are adjusted by weight coefficients. Each weight coefficient can be assigned by expert experience or determined by the analytic hierarchy process. The shape adjustment coefficient can be assigned by expert experience or calibrated by experiment.

[0068] In the above embodiments, the detection system processes the lens chromatic aberration by comparing it to the maximum permissible chromatic aberration. For example, the maximum permissible chromatic aberration can be 0.05 mm. When the actual chromatic aberration is 0.03 mm, the chromatic aberration index is 0.6. This processing converts the absolute value of the chromatic aberration into a dimensionless parameter representing the relative degree of deviation. The resolution parameter is processed by maximum-minimum normalization. For example, when the sensor resolution range is 20 megapixels to 50 megapixels, the resolution index corresponding to 40 megapixels is 0.67. This processing eliminates dimensional differences, making resolutions of different orders of magnitude comparable. A hyperbolic tangent function is used when constructing the basic state model. A non-linear transformation is applied to the color difference index. For example, when k=5, a decrease in the color difference index from 0.8 to 0.6 will cause the function value to decrease from 0.58 to 0.38. This function rapidly decays the coefficient value when the color difference approaches the allowable threshold. Simultaneously, through weighting coefficients... and The allocation, for example and This enables a graded evaluation mechanism where color difference is the dominant factor, thereby dynamically reflecting the impact of the sensor's own state on imaging quality.

[0069] Compared to existing technologies, traditional methods typically evaluate resolution alone or use fixed thresholds to determine whether color difference is acceptable, such as directly comparing resolution values ​​or setting binary criteria for color difference. This proposed solution achieves multi-parameter lateral comparability through normalization processing. Combined with nonlinear functions and weighting mechanisms, it can more accurately characterize the nonlinear degradation effect on performance when color difference approaches a critical value, while quantifying the positive contribution of resolution improvement to performance. For example, when color difference is close to but does not exceed the allowable threshold, traditional methods might deem it acceptable, while this solution uses a hyperbolic tangent function to significantly reduce the fundamental state coefficient, better reflecting the nonlinear relationship between actual image quality and color difference.

[0070] Through the technical solutions in the above embodiments, this application solves the problem in the prior art of not quantifying the impact of the sensor's own state on the detection results, and realizes a dynamic and accurate evaluation of the basic performance of visible light image sensors. By establishing a joint evaluation model of color difference and resolution, accurate sensor state parameters are provided for subsequent detection result fusion. For example, when the lens color difference index reaches 0.8, the basic state coefficient can drop to 0.75, thereby automatically reducing the confidence of visible light data in the fusion weight calculation and avoiding the risk of misjudgment due to sensor performance degradation.

[0071] In one optional embodiment, in order to determine the visible light confidence level corresponding to the UAV, the detection system first normalizes the ambient visibility and light intensity in the illumination parameters to obtain the ambient visibility index and the light intensity index. Then, the detection system performs a ratio processing on the illumination angle in the illumination parameters with a preset maximum illumination angle to obtain the illumination angle index. Finally, the detection system calculates the visible light confidence level corresponding to the UAV based on the sensor coefficient corresponding to the visible light image sensor, the ambient visibility index, the light intensity index, and the illumination angle index.

[0072] Optionally, after the detection system obtains the sensor coefficients, environmental visibility index, light intensity index, and illumination angle index corresponding to the visible light image sensor, the detection system constructs a visible light confidence model based on the illumination angle index, visibility index, and light intensity index under the basic state coefficients of the image sensor to obtain the visible light confidence level. The visible light confidence model is shown in the following formula (3):

[0073] (3)

[0074] Optionally, This represents the confidence level for visible light. Represents the fundamental state coefficients of the image sensor. Indicates the illumination angle index, Indicates visibility index, This represents the visibility index threshold. Indicates the light intensity index. Indicates the threshold of light intensity index. This represents the visibility sensitivity coefficient. Represents the light intensity sensitivity coefficient, where, Furthermore, the higher the value, the better the confidence level of the detection results from the high-resolution visible light image sensor.

[0075] Optionally, the illumination angle index refers to the ratio of the illumination angle to the maximum illumination angle. Specifically, it can be achieved by dividing the actual illumination angle by 90 degrees. This index is used to quantify the degree to which the illumination direction deviates from vertical illumination and to suppress light spots or shadow interference caused by large-angle incident light.

[0076] Optionally, the visibility index refers to the environmental visibility after maximum-minimum normalization. Specifically, it can be achieved by dividing the actual visibility by a preset maximum visibility threshold, which is used to eliminate dimensional differences and reflect the linear effect of visibility on image sharpness.

[0077] Optionally, the illuminance index refers to the illuminance value after maximum-minimum normalization. Specifically, it can be achieved by subtracting the minimum allowable intensity from the actual illuminance and then dividing by the difference between the maximum and minimum allowable intensities. This index is used to characterize the risk of overexposure or underexposure of images caused by excessively strong or weak lighting. Visibility and illuminance sensitivity coefficients can be determined through expert experience or calibrated experimentally.

[0078] Optionally, the image sensor's fundamental state coefficient refers to a comprehensive indicator reflecting the sensor's own performance. Specifically, it can be achieved through a weighted combination of the resolution normalization index and the color difference index, and is used to adjust the global impact of the sensor's fundamental performance on the visible light confidence level.

[0079] In the above embodiments, the detection system determines the illumination angle index using a cosine square function model. When the illumination angle approaches 90 degrees, the cosine term approaches zero, reducing confidence and suppressing image quality degradation caused by side illumination. The visibility index and illumination intensity index are threshold-controlled using a sigmoid function. When visibility falls below a preset threshold or illumination intensity deviates from the ideal range, the exponential function rapidly decays the confidence, avoiding misjudgments under low visibility or abnormal lighting conditions. The image sensor's fundamental state coefficient serves as a global multiplier, incorporating the sensor's resolution and color difference state into the confidence calculation. For example, when sensor resolution decreases or color difference increases, this coefficient decreases to reduce overall confidence. Through this multi-factor coupled modeling, the visible light confidence can dynamically reflect the combined influence of environmental conditions and sensor state.

[0080] Compared to existing technologies, current methods typically only use fixed thresholds to determine lighting conditions, failing to quantify the nonlinear impact of lighting angle on image quality and ignoring the dynamic changes in sensor performance. This proposed solution utilizes a composite model of the cosine square function and the sigmoid function to realize the dynamic attenuation effects of lighting angle deviation, insufficient visibility, and abnormal lighting intensity. Simultaneously, it introduces a fundamental sensor state coefficient to jointly evaluate environmental factors and hardware status, overcoming the limitations of single environmental assessments or static sensor performance assumptions.

[0081] Through the technical solutions in the above embodiments, this application can dynamically suppress image distortion caused by lateral illumination according to the actual illumination angle, avoid misjudgment under low visibility or abnormal illumination conditions through the threshold sensitivity mechanism of visibility and illumination intensity, and adjust the confidence weight in real time based on the sensor's basic state, thereby accurately evaluating the reliability of the detection results of the high-resolution visible light image sensor under different environmental conditions.

[0082] In one optional embodiment, to determine the confidence level of infrared thermal imaging corresponding to the UAV, the detection system first normalizes the resolution in the sensor parameters corresponding to the infrared thermal imaging sensor to obtain the resolution index of the infrared thermal imaging sensor. Then, the detection system calculates the thermal sensitivity index based on the thermal sensitivity and the preset minimum thermal sensitivity in the sensor parameters. Next, the detection system ratios the absolute difference between the infrared center wavelength in the sensor parameters and the preset infrared center wavelength with the preset deviation value of the infrared center wavelength to obtain the wavelength deviation index. Subsequently, the detection system calculates the confidence level of infrared thermal imaging corresponding to the UAV based on the environmental state coefficient, the resolution index of the infrared thermal imaging sensor, the thermal sensitivity index, and the wavelength deviation index.

[0083] Optionally, the detection system normalizes the resolution in the sensor parameters corresponding to the infrared thermal imaging sensor, which means linearly transforming the original resolution data to the [0,1] interval to eliminate the influence of different dimensions on the resolution evaluation.

[0084] Optionally, the detection system incorporates thermal sensitivity. In the formula, the thermal sensitivity index is determined, where, Indicates thermal sensitivity. This represents the minimum standard thermal sensitivity (i.e., the preset minimum thermal sensitivity). This formula ensures that the exponent takes the maximum value of 1 when the actual thermal sensitivity is better than the standard value, thus avoiding a decrease in confidence due to low sensitivity.

[0085] Optionally, after obtaining the environmental state coefficient, the resolution index, the thermal sensitivity index, and the wavelength deviation index of the infrared thermal imaging sensor, the detection system constructs an infrared thermal imaging confidence model based on the resolution index, thermal sensitivity index, and wavelength deviation index of the infrared thermal imaging sensor under the environmental state coefficient, and outputs the infrared thermal imaging confidence. The infrared thermal imaging confidence model is shown in the following formula (4):

[0086] (4)

[0087] In the above formula (4), Indicates the confidence level of the infrared thermal imaging sensor. Represents the environmental state coefficient. This indicates the resolution index of the infrared thermal imaging sensor. Indicates thermal sensitivity index, The wavelength deviation index is represented by, where, Furthermore, the larger the value, the better the confidence level of the infrared thermal imaging sensor's detection results.

[0088] In the above embodiments, the infrared thermal imaging confidence model achieves dynamic evaluation through multi-dimensional parameter fusion. First, the resolution index eliminates dimensional differences through normalization, objectively reflecting the sensor's imaging clarity. Second, the thermal sensitivity index ensures the sensor has basic temperature difference recognition capabilities through threshold limiting, avoiding inflated confidence levels in low-sensitivity scenarios. Third, the wavelength deviation index quantifies wavelength matching through ratio calculation, suppressing imaging distortion caused by wavelength shift. Finally, the environmental state coefficient serves as a global adjustment factor, embedding the influence of environmental factors such as humidity and temperature on sensor performance into the confidence calculation. By multiplying and exponentially decaying the resolution index, thermal sensitivity index, wavelength deviation index, and environmental state coefficient, the model can dynamically output a confidence value reflecting the coupling effect between environmental conditions and sensor state, providing an accurate basis for subsequent weight allocation.

[0089] Compared to existing technologies, traditional methods typically assess the confidence level of infrared sensors based solely on fixed parameters or single environmental factors, neglecting the dynamic effects of ambient humidity and temperature on sensor resolution and thermal sensitivity, and failing to consider the degrading effect of wavelength matching on image quality. This proposed solution addresses the static and one-sided nature of existing confidence level assessments by introducing an environmental state coefficient in conjunction with multi-dimensional sensor parameter modeling. This integration of environmental factors, sensor performance indicators, and wavelength matching in the confidence level calculation solves the problems of static and one-sided confidence level assessments found in existing technologies.

[0090] Through the technical solutions in the above embodiments, this application can dynamically evaluate the detection confidence of infrared thermal imaging sensors in complex inspection scenarios. By jointly modeling environmental state coefficients and multi-dimensional sensor parameters, it can accurately reflect the impact of environmental condition changes on sensor performance, avoid confidence evaluation deviations caused by environmental factors, thereby improving the rationality of multi-sensor fusion weight allocation and enhancing the accuracy of road anomaly event identification.

[0091] In one optional embodiment, to determine the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, the detection system first performs a ratio processing on the angle difference between the target angle and the preset angle in the flight parameters, and the preset deviation angle value, to obtain the target angle deviation index. Here, the target angle is the angle formed by the line connecting the UAV and the associated target of the abnormal event and the horizon. Then, the detection system performs a ratio processing on the flight speed in the flight parameters and the preset speed to obtain the speed index. Next, the detection system performs a ratio processing on the flight altitude in the flight parameters and the preset altitude to obtain the altitude index. Subsequently, based on the target angle deviation index, the speed index, and the altitude index, the detection system calculates the visible light confidence influence factor and the infrared thermal imaging confidence influence factor. Finally, based on the visible light confidence influence factor and the infrared thermal imaging confidence influence factor, the detection system calculates the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor.

[0092] Optionally, the detection system constructs a visible light confidence influence function based on the target angle deviation index, velocity index, and height index to obtain the visible light confidence influence factor. The visible light confidence influence function is shown in the following formula (5):

[0093] (5)

[0094] Optionally, This represents the visible light confidence level influence factor. Indicates the speed index, Indicates the height index, This indicates that the target angle deviates from the index. Represents the speed sensitivity coefficient. Indicates a high sensitivity coefficient. The angle sensitivity coefficient can be assigned a value through expert experience or calibrated through experiments. Furthermore, the larger the value, the smaller the negative impact on the quality of visible light images.

[0095] Optionally, the detection system constructs an infrared thermal imaging confidence influence function based on the target angle deviation index, velocity index, and height index to obtain the infrared thermal imaging confidence influence factor. The infrared thermal imaging confidence influence function is shown in the following formula (6):

[0096] (6)

[0097] Optionally, This represents the confidence level impact factor for infrared thermal imaging. Indicates the speed index, Indicates the height index, This indicates that the target angle deviates from the index. Indicates the angular attenuation coefficient. Furthermore, the larger the value, the smaller the impact on infrared image quality.

[0098] Optionally, the detection system constructs a dynamic weight allocation model based on the visible light confidence influence factor, the infrared thermal imaging confidence influence factor, the visible light confidence, and the infrared thermal imaging confidence, and outputs the respective weights of the detection results of the high-resolution visible light image sensor and the infrared thermal imaging sensor in the detection result fusion model. The dynamic weight allocation model is shown in the following formula (7):

[0099] (7)

[0100] In the above formula (7), This represents the weight of the detection results from the high-resolution visible light image sensor in the detection result fusion model. This indicates the weight of the detection results from the infrared thermal imaging sensor in the detection result fusion model. This represents the visible light confidence level influence factor. This represents the confidence level impact factor for infrared thermal imaging. This represents the confidence level for visible light. This indicates the confidence level of infrared thermal imaging.

[0101] Optionally, the target angle deviation index refers to the degree to which the angle between the line connecting the UAV and the target and the ground deviates from the optimal angle (i.e., the preset angle). Specifically, it can be calculated by the ratio of the difference between the actual angle and the optimal angle to the allowable deviation value, and is used to quantify the negative impact of the pitch angle on image quality.

[0102] Optionally, the speed index is the ratio of flight speed to a reference speed, specifically calculated by dividing the actual speed by a preset standard speed, reflecting the blurring effect of speed on imaging. The altitude index is the ratio of flight altitude to a reference altitude, specifically calculated by dividing the actual altitude by a standard altitude, characterizing the degree of image resolution degradation caused by altitude changes.

[0103] Optionally, the visible light confidence influence function is a function formed by taking the reciprocal of the weighted sum of the squares of the velocity, altitude and angle deviation exponents. Specifically, it can be constructed using the reciprocal form of the quadratic summation. It is used to suppress the interference of visible light image quality degradation at high speed, high flight or large pitch angle on the fusion result. Each sensitivity coefficient can be determined by expert experience or calibrated experimentally.

[0104] Optionally, the infrared thermal imaging confidence influence function is a function formed by combining an exponential decay term of the product of velocity and altitude with an S-shaped function of angle deviation. Specifically, it can be constructed by the product of an exponential function and a logical function. It is used to reduce the thermal noise interference of infrared images under high-speed and high-flight conditions and adjust the sensitivity of angle deviation. The angle decay coefficient can be assigned by expert experience or calibrated by experiment.

[0105] Optionally, the dynamic weight allocation model refers to a calculation model that normalizes the confidence scores and their influencing factors of visible light and infrared light. Specifically, it can be implemented as the ratio of the product of the confidence score and the influencing factor, and is used to adaptively adjust the fusion weights of the two types of sensors.

[0106] In the above embodiments, the detection system collects real-time data on the UAV's pitch angle, flight speed, and altitude, and calculates the target angle deviation index, speed index, and altitude index, respectively. The speed index, altitude index, and angle deviation index are input into the visible light confidence influence function to obtain a dynamically changing visible light confidence influence factor that varies with flight status. This factor decreases as the flight parameter deviation increases, thereby reducing the weight of the visible light sensor under adverse conditions. Simultaneously, the same parameters are input into the infrared thermal imaging confidence influence function. An exponential decay term suppresses the combined negative impact of speed and altitude, and an S-shaped function controls the magnitude of the angle deviation's influence, resulting in an infrared thermal imaging confidence influence factor. Finally, the two types of influence factors are normalized with the pre-calculated visible light confidence and infrared thermal imaging confidence to generate dynamic fusion weights, ensuring that sensor data less affected by interference is prioritized when the UAV is moving at high speed or pitching at large angles.

[0107] Compared to existing technologies, traditional methods employing fixed weights or simple fusion strategies based on single environmental parameters are unable to address image quality fluctuations caused by changes in flight conditions. This proposed solution overcomes the insufficient adaptability of fixed fusion ratios in complex motion scenarios by establishing a quantitative relationship model between flight parameters and sensor performance and incorporating dynamic impact assessments of speed, altitude, and pitch angle into the weight calculation. Existing technologies do not consider the cumulative effect of flight speed on the thermal noise of infrared thermal imaging sensors; this solution effectively suppresses the negative impact of infrared image quality degradation at high speeds and altitudes on the fusion results through an exponential decay term in the product of speed and altitude.

[0108] Through the technical solutions in the above embodiments, this application achieves dynamic optimization of multi-sensor fusion weights during UAV inspection. It automatically reduces the weight ratio of affected sensors when there are sudden changes in flight speed, altitude adjustments, or pitch angle deviations, solving the problem of inaccurate fusion decisions caused by neglecting flight state parameters in traditional methods. This solution can accurately evaluate the effective detection capabilities of visible light and infrared sensors based on real-time flight data. In road debris identification scenarios, it reduces the visible light weight when the UAV flies at high speed over the target area to avoid misjudgments caused by motion blur. In traffic congestion monitoring scenarios, it automatically increases the decision weight of the visible light sensor when a sudden drop in flight altitude causes a decrease in infrared thermal imaging resolution, ensuring detection reliability under complex conditions.

[0109] In one optional embodiment, in the process of controlling a visible light image sensor and an infrared thermal imaging sensor based on sensor weights to detect the event type of an abnormal event occurring on a target road and obtain the target event type, the detection system first constructs a preset event type set, wherein the preset event type in the preset event type set is one of the following: spilled material, traffic accident, traffic congestion, road damage; then, the detection system controls the visible light image sensor to detect the probability that the abnormal event belongs to each preset event type in the preset event type set, obtaining a first occurrence probability corresponding to each preset event type; subsequently, the detection system controls the infrared thermal imaging sensor to detect the probability that the abnormal event belongs to each preset event type in the preset event type set, obtaining a second occurrence probability corresponding to each preset event type; then, based on the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, the detection system performs a weighted sum of the first occurrence probability and the second occurrence probability corresponding to each preset event type to obtain the target occurrence probability corresponding to each preset event type; finally, the detection system selects the preset event type with the highest target occurrence probability in the preset event type set as the target event type.

[0110] Optionally, the detection system determines the probability of an abnormal event belonging to a preset event type based on a weighted model of detection results. The detection result fusion weight model is shown in the following formula (8):

[0111] (8)

[0112] In the above formula (8), This indicates that the merged abnormal event belongs to a category. The final probability, the and , This indicates that the target predicted by the high-resolution visible light image sensor belongs to the category. The probability, This indicates that the infrared thermal imaging sensor predicts the target belongs to The probability of the category, This represents the weight of the detection results from the high-resolution visible light image sensor in the detection result fusion model. This indicates the weight of the detection results from the infrared thermal imaging sensor in the detection result fusion model.

[0113] Optionally, the target category set It refers to the set of abnormal event types that need to be detected during road inspections, specifically limited to four categories: spilled objects, traffic accidents, traffic congestion, and road surface damage, thereby eliminating interference from irrelevant targets through category constraints.

[0114] Optionally, the high-resolution visible light image sensor predicts that the target belongs to a category. The probability is obtained by inputting the high-resolution images it collects into a multi-class image classification model pre-trained based on a deep convolutional neural network, and the normalized class probability distribution is directly output by the Softmax function (a function used to transform a set of real values ​​into a set of probability values) at the end of the model.

[0115] Optionally, the infrared thermal imaging sensor predicts that the target belongs to a category. The probability of classification is obtained by inputting the collected thermal radiation image into a multi-class classification model with a similar structure to the above network but independently trained on an infrared dataset, and the probability of each class is obtained through the Softmax function.

[0116] Optionally, a dynamic weight allocation model refers to a computational model that adjusts the weights of visible light and infrared sensors in real time based on sensor status, environmental parameters, and UAV flight parameters. Specifically, it can be implemented using a linear weighting formula combined with a dynamic influence factor function. This dynamic weight allocation model can adapt to sensor performance fluctuations in different scenarios. (Classification probability) and It refers to the target category prediction probability value output by visible light image sensors and infrared thermal imaging sensors, which can be implemented using the Softmax output layer of a deep learning classification model, and is used to quantify the confidence of different sensors in identifying the target category.

[0117] In the above embodiments, the detection system calculates the fusion weights of the visible light and infrared sensors in real time using a dynamic weight allocation model, and performs weighted fusion by combining the classification probabilities of the two. During operation, the weights of the visible light sensor... Weights of infrared sensors The classification probability is dynamically adjusted based on sensor performance indicators, environmental conditions, and UAV flight parameters. For example, in low-light or high-humidity environments, the visible light weight is reduced and the infrared weight is increased. and Generated by visible light image classification models and infrared thermal imaging classification models respectively, and fused using a probability fusion formula. Output the final recognition result. The target category set C is limited to four types of road anomalies. A category filtering mechanism is used to avoid interference from other irrelevant targets in the fusion result, such as excluding misclassification of non-road anomalies such as vegetation and animals.

[0118] Compared to existing technologies, current methods typically employ fixed-weight fusion or simple weight adjustments based on a single environmental parameter, such as assigning weights solely based on illumination intensity. These methods fail to address the coupled effects of multiple factors, including changes in UAV speed and pitch angle shifts. This proposed solution utilizes a dynamic weight allocation model to comprehensively evaluate sensor performance, environmental conditions, and flight parameters. For example, it automatically reduces the visible light weight when high-speed UAV flight causes image blurring, and adjusts the infrared weight when the pitch angle deviates from the optimal angle, thus achieving adaptive fusion of multi-source data. Furthermore, existing technologies do not explicitly define the target category range, making them susceptible to interference from non-road anomalies. This solution, however, enhances the specificity of the fusion results by pre-setting a category set.

[0119] Through the technical solutions of the above embodiments, this application solves the problem of insufficient accuracy in identifying abnormal road events caused by fixed weight allocation, and realizes dynamic optimization and fusion of detection results from visible light and infrared sensors. Specific effects include: improving the reliability of fusion results by dynamically adjusting weights under complex environmental conditions, such as prioritizing infrared sensor data in hazy weather; reducing misjudgment of non-road abnormal targets through category constraint mechanisms, such as avoiding misjudging tree shadows beside the road as debris; and maintaining stable recognition performance when the UAV's flight state changes abruptly, such as reducing the impact of image quality degradation through weight compensation during high-speed flight or large-angle pitch.

[0120] In one optional embodiment, a method for intelligent identification of unmanned aerial vehicle (UAV) road inspection based on a big data model is provided. Figure 2 This is a flowchart of an optional intelligent identification method for UAV road inspection based on a big data model, according to an embodiment of this application. Figure 2As shown, the method includes: constructing an environmental state model based on ambient humidity and ambient temperature, and outputting environmental state coefficients; constructing a basic state model of an image sensor based on the resolution of a high-resolution visible light image sensor and lens chromatic aberration, and outputting basic state coefficients of the image sensor; constructing a visible light confidence model based on ambient visibility, light intensity, and light angle under the basic state coefficients of the image sensor, and outputting visible light confidence; constructing an infrared thermal imaging confidence model based on the resolution, thermal sensitivity, and infrared center wavelength of an infrared thermal imaging sensor under the environmental state coefficients, and outputting infrared thermal imaging confidence; and constructing a dynamic weight allocation model based on the visible light confidence and infrared thermal imaging confidence at target angle, UAV movement speed, and altitude, and outputting the respective weights of the detection results of the high-resolution visible light image sensor and the infrared thermal imaging sensor in the detection result fusion model.

[0121] As described above, this technical solution constructs an environmental state model and an image sensor basic state model to evaluate the environment and sensor performance in real time, thereby improving the reliability of the data source. It also utilizes a visible light confidence model and an infrared thermal imaging confidence model, combined with factors such as environmental visibility, light intensity, and thermal sensitivity, to accurately calculate the confidence of each sensor. Finally, based on the UAV's flight state (target angle, speed, altitude), it dynamically adjusts the fusion weights of the visible light and infrared sensors, optimizing the detection result fusion process and improving the accuracy and robustness of target recognition. This approach is applicable to various abnormal road scenarios, such as spilled debris, traffic accidents, traffic congestion, and road surface damage, enhancing the system's practicality and generalization capabilities.

[0122] Example 2

[0123] This application embodiment can also provide an event detection device based on UAV road inspection. It should be noted that the event detection device based on UAV road inspection in this application embodiment can be used to execute the event detection method based on UAV road inspection provided in this application embodiment. The event detection device based on UAV road inspection provided in this application embodiment will be described below.

[0124] According to an embodiment of this application, an apparatus for implementing the above-described event detection method based on unmanned aerial vehicle (UAV) road inspection is also provided. Figure 3 This is a schematic diagram of an optional event detection device based on unmanned aerial vehicle (UAV) road inspection according to an embodiment of this application, such as... Figure 3 As shown, the device includes: a parameter acquisition unit 301, a visible light confidence determination unit 302, an infrared thermal imaging confidence determination unit 303, a sensor weight determination unit 304, and a target detection unit 305.

[0125] Optionally, the parameter acquisition unit 301 is used to acquire the environmental state coefficient, illumination parameters, and flight parameters of the target road inspected by the UAV, wherein the environmental state coefficient is used to characterize the temperature and humidity information of the target road; the visible light confidence determination unit 302 is used to determine the visible light confidence of the UAV based on the sensor coefficient and illumination parameters corresponding to the visible light image sensor in the UAV, wherein the sensor coefficient is used to characterize the resolution information and lens chromatic aberration information of the visible light image sensor; the infrared thermal imaging confidence determination unit 303 is used to determine the infrared thermal imaging confidence of the UAV based on the sensor parameters and environmental state coefficient corresponding to the infrared thermal imaging sensor in the UAV, wherein the sensor parameters include at least the resolution, thermal sensitivity, and infrared center wavelength of the infrared thermal imaging sensor; the sensor weight determination unit 304 is used to determine the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor based on the flight parameters, the visible light confidence, and the infrared thermal imaging confidence; and the target detection unit 305 is used to control the visible light image sensor and the infrared thermal imaging sensor based on the sensor weights to detect the event type of abnormal events occurring in the target road and obtain the target event type.

[0126] In one optional embodiment, the parameter acquisition unit 301 includes: a humidity deviation index determination subunit, a temperature deviation index determination subunit, and an environmental state coefficient calculation subunit.

[0127] Optionally, the humidity deviation index determination subunit is used to calculate the humidity deviation index by comparing the absolute difference between the ambient humidity and the preset humidity with the preset humidity deviation value; the temperature deviation index determination subunit is used to calculate the temperature deviation index by comparing the absolute difference between the ambient temperature and the preset temperature with the preset temperature deviation value; and the environmental state coefficient calculation subunit is used to calculate the environmental state coefficient of the target road based on the humidity deviation index and the temperature deviation index.

[0128] In one optional embodiment, the event detection device based on UAV road inspection further includes: a resolution index determination unit, a color difference index determination unit, and a sensor coefficient calculation unit.

[0129] Optionally, the resolution index determination unit is used to normalize the resolution of the visible light image sensor to obtain the resolution index of the visible light image sensor; the color difference index determination unit is used to process the ratio of the lens color difference of the visible light image sensor to the preset maximum lens color difference to obtain the color difference index; and the sensor coefficient calculation unit is used to calculate the sensor coefficient corresponding to the visible light image sensor based on the resolution index and color difference index of the visible light image sensor.

[0130] In an optional embodiment, the visible light confidence determination unit 302 further includes: a first normalization processing subunit, an illumination angle index determination subunit, and a visible light confidence determination subunit.

[0131] Optionally, the first normalization processing subunit is used to normalize the ambient visibility and light intensity in the illumination parameters to obtain the ambient visibility index and the light intensity index; the illumination angle index determination subunit is used to process the illumination angle in the illumination parameters by ratio with the preset maximum illumination angle to obtain the illumination angle index; and the visible light confidence determination subunit is used to calculate the visible light confidence of the UAV based on the sensor coefficients corresponding to the visible light image sensor, the ambient visibility index, the light intensity index, and the illumination angle index.

[0132] In an optional embodiment, the infrared thermal imaging confidence determination unit 303 further includes: a second normalization processing subunit, a thermal sensitivity index determination subunit, a wavelength deviation index determination subunit, and an infrared thermal imaging confidence determination subunit.

[0133] Optionally, the second normalization processing subunit is used to normalize the resolution in the sensor parameters corresponding to the infrared thermal imaging sensor to obtain the resolution index of the infrared thermal imaging sensor; the thermal sensitivity index determination subunit is used to calculate the thermal sensitivity index based on the thermal sensitivity in the sensor parameters and the preset minimum thermal sensitivity; the wavelength deviation index determination subunit is used to perform ratio processing on the absolute difference between the infrared center wavelength in the sensor parameters and the preset infrared center wavelength and the preset deviation infrared center wavelength value to obtain the wavelength deviation index; and the infrared thermal imaging confidence determination subunit is used to calculate the infrared thermal imaging confidence of the UAV based on the environmental state coefficient, the resolution index of the infrared thermal imaging sensor, the thermal sensitivity index, and the wavelength deviation index.

[0134] In one optional embodiment, the sensor weight determination unit 304 includes: an angle deviation index determination subunit, a speed index determination subunit, a height index determination subunit, an influence factor calculation subunit, and a sensor weight calculation subunit.

[0135] Optionally, the angle deviation index determination subunit is used to process the angle difference between the target angle and the preset angle in the flight parameters and the preset deviation angle value to obtain the target angle deviation index, where the target angle is the angle formed by the line connecting the UAV and the associated target of the abnormal event and the horizon; the speed index determination subunit is used to process the flight speed in the flight parameters and the preset speed to obtain the speed index; the altitude index determination subunit is used to process the flight altitude in the flight parameters and the preset altitude to obtain the altitude index; the influence factor calculation subunit is used to calculate the visible light confidence influence factor and the infrared thermal imaging confidence influence factor based on the target angle deviation index, the speed index, and the altitude index; the sensor weight calculation subunit is used to calculate the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively based on the visible light confidence influence factor and the infrared thermal imaging confidence influence factor.

[0136] In one optional embodiment, the target detection unit 305 includes: a type set construction subunit, a first occurrence probability detection subunit, a second occurrence probability detection subunit, a target occurrence probability detection subunit, and a target event type determination subunit.

[0137] Optionally, a type set construction subunit is used to construct a preset event type set, wherein the preset event type in the preset event type set is one of the following: spilled material, traffic accident, traffic congestion, road damage; a first occurrence probability detection subunit is used to control a visible light image sensor to detect the probability that an abnormal event belongs to each preset event type in the preset event type set, and obtain a first occurrence probability corresponding to each preset event type; a second occurrence probability detection subunit is used to control an infrared thermal imaging sensor to detect the probability that an abnormal event belongs to each preset event type in the preset event type set, and obtain a second occurrence probability corresponding to each preset event type; a target occurrence probability detection subunit is used to perform a weighted sum of the first occurrence probability and the second occurrence probability corresponding to each preset event type based on the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, and obtain a target occurrence probability corresponding to each preset event type; a target event type determination subunit is used to select the preset event type with the highest target occurrence probability in the preset event type set as the target event type.

[0138] It should be noted that the parameter acquisition unit 301, visible light confidence determination unit 302, infrared thermal imaging confidence determination unit 303, sensor weight determination unit 304, and target detection unit 305 mentioned above correspond to steps S101 to S105 in the method embodiment. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in the above embodiment.

[0139] Example 3

[0140] Embodiments of this application can also provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, the electronic device includes: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0141] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and devices in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned event detection method based on UAV road inspection.

[0142] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0143] The processor can access information and applications stored in memory via a transmission device to execute the following steps: acquiring the environmental state coefficients, illumination parameters, and flight parameters of the target road being inspected by the UAV, where the environmental state coefficients characterize the temperature and humidity information of the target road; determining the visible light confidence level of the UAV based on the sensor coefficients and illumination parameters corresponding to the visible light image sensor in the UAV, where the sensor coefficients characterize the resolution and lens chromatic aberration information of the visible light image sensor; determining the infrared thermal imaging confidence level of the UAV based on the sensor parameters and environmental state coefficients corresponding to the infrared thermal imaging sensor in the UAV, where the sensor parameters include at least the resolution, thermal sensitivity, and infrared center wavelength of the infrared thermal imaging sensor; determining the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor based on the flight parameters, visible light confidence level, and infrared thermal imaging confidence level; and controlling the visible light image sensor and the infrared thermal imaging sensor based on the sensor weights to detect the event types of abnormal events occurring on the target road, thereby obtaining the target event type.

[0144] This application provides a scheme for event detection based on UAV road inspection. Based on environmental state coefficients, this application quantifies the impact of temperature and humidity information in the road inspection environment on the performance of the infrared thermal imaging sensor in the UAV, obtaining the infrared thermal imaging confidence level. Simultaneously, based on illumination parameters, this application quantifies the impact of illumination information in the road inspection environment on the performance of the visible light sensor in the UAV, obtaining the visible light confidence level. This achieves the goal of accurately evaluating both the visible light confidence level and the infrared thermal imaging confidence level.

[0145] Subsequently, based on the UAV's flight parameters, visible light confidence level, and infrared thermal imaging confidence level, this application adjusts the sensor weights used by the visible light image sensor and the infrared thermal imaging sensor when fusing detection results. This achieves adaptive fusion of detection data from multiple sensors under different road inspection environments and UAV flight conditions, thereby improving the detection accuracy of abnormal events occurring on inspected roads. It also solves the technical defects of rigid weight allocation and inaccurate weight allocation results caused by the lack of environmental interference quantification and incomplete sensor state evaluation during the fusion of detection data from multiple sensors. Furthermore, it solves the technical problem of inaccurate detection results of abnormal events on roads during road inspections based on existing technologies.

[0146] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, PDAs, mobile internet devices, PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0148] Example 4

[0149] Embodiments of this application may also provide a storage medium.

[0150] Optionally, in this embodiment of the application, the storage medium can be used to store the program code executed by the event detection method based on UAV road inspection provided in the above method embodiment.

[0151] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0152] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing the steps of an event detection method based on unmanned aerial vehicle (UAV) road inspection.

[0153] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0154] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0159] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An event detection method based on unmanned aerial vehicle (UAV) road inspection, characterized in that, include: The environmental state coefficient, illumination parameters, and flight parameters of the target road inspected by the UAV are obtained, wherein the environmental state coefficient is used to characterize the temperature and humidity information of the target road. Based on the sensor coefficients corresponding to the visible light image sensor in the UAV and the illumination parameters, the visible light confidence level of the UAV is determined, wherein the sensor coefficients are used to characterize the resolution information and lens chromatic aberration information of the visible light image sensor; Based on the sensor parameters corresponding to the infrared thermal imaging sensor in the UAV and the environmental state coefficient, the infrared thermal imaging confidence level of the UAV is determined, wherein the sensor parameters include at least the resolution, thermal sensitivity and infrared center wavelength of the infrared thermal imaging sensor. Based on the flight parameters, the visible light confidence level, and the infrared thermal imaging confidence level, the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor are determined respectively. Based on the sensor weights, the visible light image sensor and the infrared thermal imaging sensor are controlled to detect the event types of abnormal events occurring in the target road, and the target event type is obtained.

2. The event detection method based on UAV road inspection according to claim 1, characterized in that, Obtain the environmental state coefficients of the target road inspected by the drone, including: The humidity deviation index is obtained by comparing the absolute difference between the ambient humidity and the preset humidity with the preset humidity deviation value. The temperature deviation index is obtained by comparing the absolute difference between the ambient temperature and the preset temperature with the preset temperature deviation value. The environmental state coefficient of the target road is calculated based on the humidity deviation index and the temperature deviation index.

3. The event detection method based on UAV road inspection according to claim 1, characterized in that, Before determining the visible light confidence level of the UAV based on the sensor coefficients corresponding to the visible light image sensor in the UAV and the illumination parameters, the event detection method based on UAV road inspection further includes: The resolution of the visible light image sensor is normalized to obtain the resolution index of the visible light image sensor. The color difference index is obtained by comparing the lens chromatic aberration of the visible light image sensor with a preset maximum lens chromatic aberration. Based on the resolution index and the color difference index of the visible light image sensor, the sensor coefficient corresponding to the visible light image sensor is calculated.

4. The event detection method based on UAV road inspection according to claim 1, characterized in that, Based on the sensor coefficients corresponding to the visible light image sensor in the UAV and the illumination parameters, the visible light confidence level of the UAV is determined, including: The ambient visibility and light intensity in the illumination parameters are normalized to obtain the ambient visibility index and the light intensity index. The illumination angle in the illumination parameters is compared with the preset maximum illumination angle to obtain the illumination angle index. Based on the sensor coefficients corresponding to the visible light image sensor, the environmental visibility index, the light intensity index, and the light angle index, the visible light confidence level of the UAV is calculated.

5. The event detection method based on UAV road inspection according to claim 1, characterized in that, Based on the sensor parameters corresponding to the infrared thermal imaging sensor in the UAV and the environmental state coefficient, the infrared thermal imaging confidence level corresponding to the UAV is determined, including: The resolution in the sensor parameters corresponding to the infrared thermal imaging sensor is normalized to obtain the resolution index of the infrared thermal imaging sensor. The thermal sensitivity index is calculated based on the thermal sensitivity and the preset minimum thermal sensitivity in the sensor parameters. The absolute difference between the infrared center wavelength and the preset infrared center wavelength in the sensor parameters is compared with the preset deviation value of the infrared center wavelength to obtain the wavelength deviation index. Based on the environmental state coefficient, the resolution index of the infrared thermal imaging sensor, the thermal sensitivity index, and the wavelength deviation index, the infrared thermal imaging confidence level corresponding to the UAV is calculated.

6. The event detection method based on UAV road inspection according to claim 1, characterized in that, Based on the flight parameters, the visible light confidence level, and the infrared thermal imaging confidence level, the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor are determined, including: The angle difference between the target angle and the preset angle in the flight parameters is compared with the preset deviation angle value to obtain the target angle deviation index, wherein the target angle is the angle formed by the line connecting the UAV and the associated target of the abnormal event and the horizon. The ratio of the flight speed to the preset speed in the flight parameters is processed to obtain the speed index; The altitude index is obtained by comparing the flight altitude with the preset altitude in the flight parameters. Based on the target angle deviation index, the velocity index, and the height index, the visible light confidence influence factor and the infrared thermal imaging confidence influence factor are calculated. Based on the visible light confidence influence factor and the infrared thermal imaging confidence influence factor, the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor are calculated respectively.

7. The event detection method based on UAV road inspection according to claim 1, characterized in that, Based on the sensor weights, the visible light image sensor and the infrared thermal imaging sensor are controlled to detect the event types of abnormal events occurring in the target road, and the target event types are obtained, including: Construct a preset event type set, wherein the preset event type in the preset event type set is one of the following: Spilled debris, traffic accidents, traffic congestion, road damage; The visible light image sensor is controlled to detect the probability that the abnormal event belongs to each preset event type in the preset event type set, and a first occurrence probability corresponding to each preset event type is obtained; The infrared thermal imaging sensor is controlled to detect the probability that the abnormal event belongs to each preset event type in the preset event type set, and a second occurrence probability corresponding to each preset event type is obtained. Based on the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor, the first occurrence probability and the second occurrence probability corresponding to each preset event type are weighted and summed to obtain the target occurrence probability corresponding to each preset event type. The preset event type with the highest probability of occurrence in the preset event type set is taken as the target event type.

8. An event detection device based on unmanned aerial vehicle (UAV) road inspection, characterized in that, include: The parameter acquisition unit is used to acquire the environmental state coefficient, illumination parameters, and flight parameters of the target road inspected by the UAV, wherein the environmental state coefficient is used to characterize the temperature and humidity information of the target road. The visible light confidence determination unit is used to determine the visible light confidence of the UAV based on the sensor coefficients corresponding to the visible light image sensor in the UAV and the illumination parameters, wherein the sensor coefficients are used to characterize the resolution information and lens chromatic aberration information of the visible light image sensor; An infrared thermal imaging confidence determination unit is used to determine the infrared thermal imaging confidence of the UAV based on the sensor parameters corresponding to the infrared thermal imaging sensor in the UAV and the environmental state coefficient, wherein the sensor parameters include at least the resolution, thermal sensitivity and infrared center wavelength of the infrared thermal imaging sensor. The sensor weight determination unit is used to determine the sensor weights corresponding to the visible light image sensor and the infrared thermal imaging sensor respectively based on the flight parameters, the visible light confidence level, and the infrared thermal imaging confidence level. The target detection unit is used to control the visible light image sensor and the infrared thermal imaging sensor based on the sensor weights, detect the event type of abnormal events occurring in the target road, and obtain the target event type.

9. A computer program product, characterized in that, The computer program product includes a computer program, wherein, when the computer program is executed, it controls the computer program product to perform the event detection method based on UAV road inspection as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the event detection method based on UAV road inspection as described in any one of claims 1 to 7.

Images