Unmanned aerial vehicle detection method and device, electronic equipment and storage medium

Abstract

The invention relates to an unmanned aerial vehicle detection method and device, electronic equipment and a storage medium. The detection method of the unmanned aerial vehicle comprises the following steps: detecting by using a current optical imaging mode of a group of multi-dimensional variable optical imaging modes to obtain current detection information, wherein the group of multi-dimensional variable optical imaging modes are associated with spatial information, spectral information and polarization information; selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes based on the current detection information, the target imaging mode corresponding to the specific spatial information, spectral information and polarization information; and processing the current detection information by using the target imaging mode to output target detection information. In this way, the unmanned aerial vehicle has the high environment adaptability of target intelligent false target detection, smoke disguising recognition and the like, and can still autonomously obtain high-precision detection information under the environment conditions of complex scenes, false targets or disguising targets, smoke and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method, device, electronic equipment and storage medium for detecting an unmanned autonomous aerial vehicle. Background Art

[0002] The mission execution of unmanned aerial vehicles is inseparable from the accurate detection and perception information of the scene. Its detection methods can be divided into two methods: non-autonomous and autonomous. The non-autonomous method is generally for people to manually identify useful information in the scene in the loop. In the non-autonomous method, the perception of external information of the unmanned aerial vehicle depends on the ground station operator. In the flight environment without link signals or complex scenes, the mission execution of the unmanned aerial vehicle is greatly affected. This requires the unmanned aerial vehicle to be able to completely rely on its own detection equipment to complete the detection and perception tasks, that is, autonomous detection and perception. Artificial intelligence target recognition is the main autonomous perception method. In the environment of complex target scenes, false targets, camouflaged targets, smoke, etc., the autonomous detection and recognition capabilities are greatly reduced, and therefore cannot fully meet the needs of actual applications. When unmanned aerial vehicles perform tasks independently in complex scenes, there is no effective detection and perception technology and method that can meet the high-precision autonomous detection and perception requirements required for navigation.

[0003] Therefore, it is necessary to propose an unmanned autonomous aerial vehicle detection method, device, electronic equipment and storage medium, so as to achieve an autonomous optical imaging detection and perception method that can autonomously obtain high-precision detection information under complex scenes, false targets or camouflaged targets, smoke and other environmental conditions. Summary of the invention

[0004] Embodiments of the present invention provide an unmanned autonomous aerial vehicle detection method, device, electronic device and storage medium to at least partially solve the technical problems existing in the prior art.

[0005] According to a first aspect of the present invention, a method for detecting an unmanned autonomous aerial vehicle is provided. The method comprises: obtaining current detection information by detecting a current optical imaging mode of a set of multidimensional variable optical imaging modes, wherein the set of multidimensional variable optical imaging modes is associated with spatial information, spectral information, and polarization information; based on the current detection information, selecting a target imaging mode from the set of multidimensional variable optical imaging modes, wherein the target imaging mode corresponds to specific spatial information, spectral information, and polarization information; and processing the current detection information by using the target imaging mode to output target detection information.

[0006] In one embodiment, based on the current detection information, selecting a target imaging mode from the set of multidimensional variable optical imaging modes includes: using a first field of view switching module and a second field of view switching module to obtain a spatial resolution corresponding to the selected target imaging mode; and determining the target imaging mode based on the spatial resolution.

[0007] In one embodiment, based on the current detection information, selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes includes: performing band and time fusion on the current detection information to enhance the current detection information.

[0008] In one embodiment, the method further includes: using a target detection network to obtain at least one of the current detection information and the target detection information, wherein the target detection network includes an input terminal Input, a backbone network Backbone, a Neck network and a prediction terminal Prediction, wherein the input terminal Input includes Mosaic data enhancement, image size scaling and adaptive anchor frame calculation, and wherein the backbone network Backbone includes Focus processing, CBL processing, CSP structure and SPP spatial pyramid pooling.

[0009] In one embodiment, based on the current detection information, selecting a target imaging mode from the set of multidimensional variable optical imaging modes includes: inputting formatted parameters of the set of multidimensional variable optical imaging modes; inputting the current detection information; outputting formatted parameters of the target imaging mode; and processing the formatted parameter output based on a reinforcement learning intelligent decision network to obtain the target imaging mode.

[0010] In one embodiment, the intelligent decision network is trained in the following manner: randomly initializing the policy network and the value network; using the policy to interact in the environment to obtain state, action, and reward pairs; using the derived actor-critic network parameter update calculation formula as training data to update the value network; using the gradient descent method, using the defined advantage function, and using the derived Actor network parameter update calculation formula to update the policy network; and if the maximum number of iterations is reached, the training phase ends; if the maximum number of iterations is not reached, returning to the second step to continue interacting with the environment.

[0011] In one embodiment, the spatial information includes two-dimensional spatial information, and the polarization information includes imaging mode information in four polarization directions.

[0012] In a second aspect of the present invention, an unmanned autonomous aerial vehicle detection device is provided. The device comprises: a current detection information acquisition module, configured to obtain current detection information by detecting a current optical imaging mode of a group of multidimensional variable optical imaging modes, wherein the group of multidimensional variable optical imaging modes is associated with spatial information, spectral information and polarization information; a target imaging mode selection module, configured to select a target imaging mode from the group of multidimensional variable optical imaging modes based on the current detection information, wherein the target imaging mode corresponds to the specific spatial information, spectral information and polarization information; and a target detection information output module, configured to process the current detection information by using the target imaging mode to output target detection information.

[0013] In a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to the first aspect of the present invention.

[0014] In a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the program is executed by a processor, the method according to the first aspect of the present invention is implemented.

[0015] In a fifth aspect of the present invention, a computer program product is provided, wherein the product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method according to the first aspect of the present invention is implemented.

[0016] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present invention can be implemented;

[0019] Figure 2 A flowchart showing an example method of an unmanned autonomous aerial vehicle according to some embodiments of the present invention is shown;

[0020] Figure 3 A flow chart of an overall detection system according to some embodiments of the present invention is shown;

[0021] Figure 4 A schematic diagram of an optical field switching method according to some embodiments of the present invention is shown;

[0022] Figure 5 A schematic diagram of a large and small field of view switching structure of an optical system according to some embodiments of the present invention is shown;

[0023] Figure 6 A schematic diagram of the structure of an image processing module according to some embodiments of the present invention is shown;

[0024] Figure 7 A schematic diagram of a target detection network model according to some embodiments of the present invention is shown;

[0025] Figure 8 A schematic diagram of a traditional actor-critic framework according to some embodiments of the present invention is shown;

[0026] Fig. 9 A schematic diagram of an intelligent decision-making algorithm network according to some embodiments of the present invention is shown;

[0027] Fig.10 A schematic diagram of online learning and offline learning of an intelligent decision algorithm network according to some embodiments of the present invention is shown;

[0028] Fig.11 A schematic block diagram of an unmanned autonomous aerial vehicle detection device according to an embodiment of the present invention is shown; and

[0029] Fig.12 A block diagram of a computing device capable of implementing various embodiments of the present invention is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0031] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] Figure 1 Schematic diagram of an example environment in which various embodiments of the present invention can be implemented. Figure 1 As shown, in some embodiments, unmanned autonomous aerial vehicles, also known as drones, are aircraft that can fly without direct human control. They are widely used in reconnaissance, commercial aerial photography, agricultural monitoring, scientific research data collection, rescue and disaster response, security monitoring, personal entertainment, industrial inspection, marine research, fire scene assessment, and emergency medical supplies transportation. These drones can be equipped with a variety of sensors and equipment such as cameras, thermal imagers, multispectral sensors, radars, and laser radars (LiDAR) to perform specific tasks, and their application scope continues to expand with the continuous advancement of technology. Therefore, the embodiments of the present invention can also be applied to other future application fields.

[0035] In some embodiments, continue to refer to Figure 1, the unmanned autonomous aerial vehicle can be communicatively coupled to the computing device 101. Although shown as a separate entity, the computing device 101 can be embedded in the unmanned autonomous aerial vehicle. The computing device 101 can also be an entity external to the unmanned autonomous aerial vehicle and can communicate with the unmanned autonomous aerial vehicle via a wireless network. The computing device 101 can be any device with computing capabilities. As a non-limiting example, the computing device 101 can be any type of fixed computing device, mobile computing device, or portable computing device, including but not limited to a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a multimedia computer, a mobile phone, etc.; all or part of the components of the computing device 101 can be distributed in the cloud. The computing device 101 includes at least a processor, a memory, and other components that are typically present in a general-purpose computer to implement computing, storage, communication, control, and other functions. The computing device 101 can perform all or part of the operations according to various embodiments of the present invention, which will be described in detail below. The computing device can be used to process images detected by cameras, etc. for object perception. In the following, for ease of explanation, it will be combined with Figure 1 To describe an embodiment of the present invention.

[0036] Figure 2 A flow chart showing an example method for an unmanned autonomous aerial vehicle according to some embodiments of the present invention is shown.

[0037] In box 201, current detection information is obtained by detecting a current optical imaging mode of a set of multi-dimensional variable optical imaging modes, wherein the set of multi-dimensional variable optical imaging modes is associated with spatial information, spectral information, and polarization information. In box 202, based on the current detection information, a target imaging mode is selected from the set of multi-dimensional variable optical imaging modes, and the target imaging mode corresponds to specific spatial information, spectral information, and polarization information. In box 203, the current detection information is processed by using the target imaging mode to output target detection information.

[0038] In one embodiment, based on current detection information, selecting a target imaging mode from a set of multidimensional variable optical imaging modes includes: using a first field of view switching module and a second field of view switching module to obtain and select the spatial resolution of the target imaging mode; and determining the target imaging mode based on the spatial resolution.

[0039] In one embodiment, based on the current detection information, selecting the target imaging mode from a set of multi-dimensional variable optical imaging modes includes: performing band and time sequence fusion on the current detection information to enhance the current detection information.

[0040] In one embodiment, the method also includes: using a target detection network to obtain at least one of current detection information and target detection information, wherein the target detection network includes an input terminal Input, a backbone network Backbone, a Neck network and a prediction terminal Prediction, wherein the input terminal Input includes Mosaic data enhancement, image size scaling and adaptive anchor frame calculation, and wherein the backbone network Backbone includes Focus processing, CBL processing, CSP structure and SPP spatial pyramid pooling.

[0041] In one embodiment, based on current detection information, selecting a target imaging mode from a set of multidimensional variable optical imaging modes includes: inputting a set of parameter formatting of multidimensional variable optical imaging modes; inputting current detection information; outputting parameter formatting of the target imaging mode; and obtaining the target imaging mode by processing the parameter formatting output based on a reinforcement learning intelligent decision network.

[0042] In one embodiment, the intelligent decision network is trained in the following manner: randomly initializing the policy network and the value network; using the policy to interact in the environment to obtain state, action, and reward pairs; using the derived actor-critic network parameter update calculation formula as training data to update the value network; using the gradient descent method, using the defined advantage function, and using the derived Actor network parameter update calculation formula to update the policy network; and if the maximum number of iterations is reached, the training phase ends; if the maximum number of iterations is not reached, returning to the second step to continue interacting with the environment.

[0043] In one embodiment, the spatial information includes two-dimensional spatial information, and the polarization information includes imaging mode information in four polarization directions.

[0044] The following will combine Figure 3 Specific embodiments according to the present invention are described. Figure 3 A flow chart of an overall detection system according to some embodiments of the present invention is shown.

[0045] like Figure 3 As shown in the figure, the multidimensional variable optical imaging system of the unmanned autonomous aerial vehicle can autonomously select and switch imaging modes according to different mission requirements and environmental conditions to improve the accuracy of detecting and identifying targets. The entire system consists of two main modules: image processing module and imaging mode decision module. The image processing module includes a multidimensional variable optical imaging device, multi-mode optical imaging data fusion, time series image data fusion and target detection. Among them, the multidimensional variable optical imaging device is the data source of the system, responsible for capturing multidimensional imaging data. These data include spatial information, spectral information and polarization information, etc., providing rich original information for subsequent processing.

[0046] In this embodiment, multi-mode optical imaging data fusion can fuse multi-dimensional data acquired from an imaging device to integrate data from different sensors or different imaging modes to improve the integrity and accuracy of the information.

[0047] Furthermore, time series image data fusion can further fuse time series image data based on multi-mode data fusion. Time series data refers to image data acquired at different time points. By fusing these data, the characteristics of the target changing over time can be captured, which is particularly important for the recognition of dynamic targets. Target detection is the process of sending the fused image data into the target detection algorithm to identify and locate the target in the image. This step is the core of image processing, and its accuracy directly affects the quality of the detection results.

[0048] In such an embodiment, the imaging mode decision may include a mode parameter encoding network, a mode decision network, an update mode, and an imaging mode switching instruction.

[0049] The mode parameter encoding network is responsible for encoding different imaging mode parameters to form inputs that can be used by the decision network. These parameters may include the characteristics of the imaging mode, environmental conditions, task requirements, etc. The mode decision network is based on the encoded mode parameters. The decision network evaluates the applicability of various imaging modes and selects the most appropriate imaging mode. This process may involve machine learning algorithms to achieve intelligent mode selection. Further, the system updates the current imaging mode based on the selection of the decision network to ensure that the system can flexibly respond to different detection tasks and environmental changes. According to the updated imaging mode, the system sends a switching instruction to the multidimensional variable optical imaging device to achieve real-time switching of the imaging mode.

[0050] As a result, the system can autonomously select the most appropriate imaging mode according to task requirements and environmental conditions, improving the adaptability and flexibility of the system. Through multi-mode and time series data fusion, the system can more accurately identify and locate targets, improving the accuracy of detection results. The imaging mode decision module uses machine learning algorithms to achieve intelligent mode selection, reduce human intervention, and improve the automation of the system. Therefore, the system can be widely used in reconnaissance, environmental monitoring, traffic monitoring, disaster response and other fields. In reconnaissance, the system can select the most appropriate imaging mode according to different environments to improve the accuracy of target recognition. In environmental monitoring, the system can monitor environmental changes in real time and provide a scientific basis for decision-making. In traffic monitoring, the system can identify traffic flow and abnormal conditions to improve the efficiency of traffic management. In disaster response, the system can quickly assess the impact of disasters and provide important information for rescue operations.

[0051] In such an embodiment, spatial-spectral-polarization multidimensional variable optical imaging has a large number of optical imaging mode sets and can quickly switch between any two imaging modes. Based on the detection information of the current optical imaging mode, intelligent information fusion processing and decision-making technology is used to quickly switch to the optimal imaging mode to obtain accurate optical detection information.

[0052] Spectral detection technology improves the recognition of targets by capturing multi-dimensional information in the target scene, which can include two-dimensional spatial information and spectral information. This technology is particularly suitable for distinguishing between real and fake targets, targets and camouflage objects, and targets and surroundings because it relies on the spectral properties of matter, which vary significantly between different objects.

[0053] Spectral detection technology can provide high-resolution spectral information, so it can capture very subtle spectral changes and distinguish the spectral characteristics of different substances. Compared with traditional multi-spectral imaging, spectral detection technology can provide continuous spectral data, which helps to analyze the spectral characteristics of the target more accurately.

[0054] In the visible light band, spectral detection technology can set full-color and multi-spectral imaging modes. Full-color imaging mode covers the wavelength range of 450nm to 900nm, providing grayscale images with high spatial resolution. Multi-spectral imaging mode can subdivide this range into three spectral bands:

[0055] Multi-spectral 1 band: 450nm to 520nm, mainly captures green light information.

[0056] Multi-spectral 2 band: 520nm to 630nm, mainly captures red light information.

[0057] Multi-spectral 3 bands: 630nm to 690nm, mainly capturing near-infrared light information.

[0058] The switching of these spectral bands is achieved through filters. The filter switching mechanism includes a filter assembly, a support structure with a gear mechanism, a motor, an encoder and a support base. By controlling the motor, the switching mechanism can be rotated to quickly and stably cut the filter group in and out of the optical system.

[0059] Polarization imaging detection uses the characteristics of different objects having different polarization information. There is a significant difference in polarization information between artificial targets and natural backgrounds. Therefore, polarization detection can identify artificially constructed targets in complex backgrounds. Polarization imaging can adopt four polarization direction imaging modes: 0°, 45°, 90°, and 135°. Each polarization direction is achieved by replacing the polarizer.

[0060] The cameras of each band in the optical imaging system can be adjusted and combined in terms of the parameters of the spatial field of view, spectrum, and polarization angle. This flexibility means that the system can fully cover the optical characteristics of most targets, ensuring clear and accurate imaging of various targets. Through arrangement, a large number of imaging mode combinations can be obtained, thereby improving the system's ability to recognize different targets. In this way, the target recognition and detection capabilities in complex environments can be significantly improved.

[0061] Figure 4 A schematic diagram of optical field switching methods according to some embodiments of the present invention is shown. Figure 5 FIG. 4 shows a schematic diagram of a large and small field of view switching structure of an optical system according to some embodiments of the present invention. For spatial imaging mode conversion, the following can be used: Figure 4 and Figure 5 The optical field of view switching method can achieve changes in the spatial resolution of the same scene.

[0062] In one embodiment, Figure 4 As shown, the large and small field of view switching lens group includes a series of movable lenses or lens groups, and the field of view is switched by changing the relative positions between the lenses or replacing different lenses. Figure 4 The left side of the middle image shows the wide field of view mode, while the right side shows the narrow field of view mode. This design allows the system to quickly switch between wide and narrow fields of view to adapt to different observation tasks.

[0063] Figure 5 The figure shows a stereoscopic image of two different optical field of view switching devices. The device on the left is used for wide field of view observation, while the device on the right may be used for narrow field of view observation. These devices may include a rotatable lens barrel and replaceable lenses to achieve field of view switching. By rotating or replacing the lens, the system can change the size of its imaging field of view, thereby obtaining images of different resolutions.

[0064] In one embodiment, as described above, the image processing module may include multi-mode optical imaging data fusion, time-series image data fusion, and target detection. In one embodiment, specifically, a multi-dimensional optical information fusion algorithm integrates optical information from different dimensions to improve image quality and detection accuracy. This algorithm is particularly suitable for processing image sequences containing multi-dimensional spatial-spectral-polarization information, as well as time-series input logic, so as to fuse the detection images in terms of bands and time sequences. Figure 6 A schematic diagram of the structure of an image processing module according to some embodiments of the present invention is shown.

[0065] like Figure 6As shown in the figure, the core components of the algorithm include multi-dimensional optical fusion, temporal image fusion, and target detection. Specifically, multi-dimensional optical fusion involves fusing optical information from different bands. By using the VisionTransformer deep learning model based on the attention mechanism, the algorithm can learn the correlation between different bands and effectively integrate this information. Vision Transformer can process high-dimensional data and capture image features in multiple dimensions, thereby improving the image representation capability.

[0066] Based on multi-dimensional optical fusion, the algorithm further integrates temporal information. This is achieved through ConvGRU (Convolutional Gated Recurrent Unit), which can process image data and capture dynamic changes in time series. ConvGRU enhances the ability to recognize target motion and changes by learning the temporal dependencies in image sequences. The fused image is fed into the YOLOv5 (You Only Look Once version 5) real-time target detection system. YOLOv5 is known for its fast speed and high accuracy, and can quickly and accurately identify and locate targets in images.

[0067] In this way, by fusing multi-dimensional optical information, the algorithm can provide a richer scene description, thereby improving the accuracy of target detection. By integrating information from different bands, the algorithm reduces image noise and improves image clarity and contrast. The introduction of YOLOv5 ensures that the algorithm can process image data in real time, which is crucial for application scenarios that require fast response.

[0068] In another embodiment, the multi-mode imaging target detection task can be decomposed into the target detection of the single band information obtained by each imaging channel and the interactive fusion process of multiple band deep features during the detection process. The target detection network structure can be, for example, Figure 7 Schematic diagram of the target detection network model shown.

[0069] exist Figure 7 In the CNN, the target detection network can be composed of four main modules: input, backbone, neck and prediction. The input is the starting point of the target detection network, which is responsible for processing the original image data to make it suitable for further processing by the network. The input mainly includes the following steps:

[0070] Mosaic data augmentation increases the diversity of training data by stitching four different images into one large image. This method can improve the model's generalization ability to different scenes.

[0071] In order to fit the network's input requirements, the original images need to be scaled to a specific size. This usually involves adjusting the resolution of the images to ensure that they can be effectively processed by the network.

[0072] Anchor boxes are reference boxes used to predict the location of the target. Adaptive anchor box calculation means dynamically generating anchor boxes based on the size and content of the input image to improve the accuracy of detection.

[0073] The backbone network is the core of the target detection network, responsible for extracting image features, and can include the following key components:

[0074] Focus processing is used to concentrate on processing low-level features of the image in the early stages of the network. It enhances the expressiveness of features by focusing on local areas of the image.

[0075] CBL stands for Convolution, Batch Normalization, and Leaky ReLU activation functions to speed up the network training process and improve the stability of the model.

[0076] The CSP (Cross Stage Partial) network structure reduces the amount of computation by sharing features between different stages while maintaining the richness of features, which helps improve the efficiency and accuracy of the network.

[0077] SPP Spatial Pyramid Pooling: SPP (Spatial Pyramid Pooling) is a pooling technique that allows the network to capture image features at different scales, thereby improving the model's ability to detect objects of different sizes.

[0078] In this embodiment, the Neck network is located between Backbone and Prediction. Its main function is to further process and fuse features to improve the accuracy of detection. The Neck network usually includes the following steps:

[0079] Through upsampling and feature concatenation, the Neck network fuses feature maps of different scales to obtain richer feature representation.

[0080] The CSP structure is used again in Neck to further optimize the transfer and fusion of features.

[0081] The Neck network is able to process feature maps of multiple scales, which helps the model detect objects at different scales.

[0082] In this embodiment, the prediction end is the output part of the object detection network, which is responsible for generating the final detection result. This usually includes the following steps:

[0083] At each scale, convolutional layers are used to predict the class and location of the object. These convolutional layers usually have a high number of channels to capture rich feature information.

[0084] Decoding the predictions output by the network into actual object locations involves applying the predicted offsets to the anchor boxes.

[0085] Non-maximum suppression (NMS) is used to remove overlapping detection boxes and retain only the boxes that are most likely to contain the target.

[0086] In one embodiment, the multi-dimensional optical imaging mode intelligent decision algorithm obtains optimized image data by intelligently selecting the best imaging mode. The algorithm may include the following operations:

[0087] Multi-dimensional optical imaging mode parameter formatting input: At this stage, the algorithm receives the original parameters from the multi-dimensional optical imaging system, such as spatial resolution, spectral range, polarization state, etc. These parameters are formatted into a form suitable for algorithm processing, providing basic data for subsequent intelligent decision-making.

[0088] Imaging data input: The algorithm processes the actual captured imaging data, which may include image information in different bands such as visible light, infrared, ultraviolet, etc. By analyzing this data, the algorithm can evaluate the acquisition of the target in the current imaging mode.

[0089] Multi-dimensional optical imaging mode parameter formatted output: Based on the results of intelligent decision-making, the algorithm outputs the optimal imaging mode parameters. These parameters guide the imaging system to adjust its working state to adapt to the current detection task and environmental conditions.

[0090] Intelligent decision-making network framework based on reinforcement learning: In order to realize the function of the intelligent agent to select the optimal action parameters under the current mode and target acquisition, the algorithm adopts the Actor-Critic framework in reinforcement learning for training.

[0091] Figure 8 FIG. 4 shows a schematic diagram of a traditional actor-critic framework according to some embodiments of the present invention. Figure 8 As shown, the framework contains two key networks:

[0092] Action Network: Using the policy gradient reinforcement learning algorithm, the system outputs the probability distribution of each possible action in a given state. In this way, the system can select the most appropriate imaging mode according to the current environment and task requirements.

[0093] Evaluation network: Uses a value function reinforcement learning algorithm to evaluate the value of the current state, thereby helping the system understand the contribution of different imaging modes to target detection.

[0094] In order to improve efficiency and reduce the number of parameters, a common network was designed before the action network and the evaluation network. The output of the common network is used as the input of the two networks, which not only reduces repeated calculations but also speeds up the convergence of the model.

[0095] Fig. 9 A schematic diagram of an intelligent decision algorithm network according to some embodiments of the present invention is shown. In this embodiment, the network framework is used for intelligent decision-making of multi-dimensional optical imaging modes, which realizes the selection of the optimal imaging mode by combining mode parameter encoding, evaluation and decision networks and physical property libraries.

[0096] like Fig. 9 As shown in Figure 1, the mode parameter encoding network is responsible for preprocessing the input imaging mode parameters. It includes multi-source parameter fusion, which integrates imaging parameters from different sources (such as wavelength, spectrum, field of view, polarization angle, etc.). At the same time, it also models the temporal relationship to understand the changes of imaging parameters over time, which is crucial for imaging of dynamic scenes.

[0097] After encoding the pattern parameters, the network combines the encoded features with other features of the imaging data through feature concatenation. This step helps enrich the feature representation of the imaging data and provides more comprehensive information for subsequent evaluation and decision-making.

[0098] The evaluation network can be a convolutional neural network (CNN), which is used to evaluate the quality or value of the image in the current imaging mode. It is used to output a value score, which indicates the effectiveness of the current imaging mode for target detection. This value score is part of the advantage function and is used to guide the update of network parameters.

[0099] The decision network is also a CNN, which determines the best imaging mode based on the output of the evaluation network. Through the cascade network structure, the decoding outputs specific mode selection parameters, which guide the imaging system on how to adjust its working state to adapt to the current detection task. The output of the decision network is the anti-interference action, that is, how to adjust the imaging mode to obtain the best image when facing complex environments or interference.

[0100] The physical property library contains the physical properties under different imaging modes, such as target reflection characteristics and polarization characteristics under different wavelengths, etc. This library provides the necessary physical background knowledge for the decision network to help it make more accurate decisions.

[0101] Update configuration Based on the output of the decision network, the configuration of the imaging system is updated to achieve the selected imaging mode. This process involves adjusting the physical parameters of the imaging system, such as switching filters and adjusting polarizers.

[0102] Images captured in a new imaging mode are evaluated again by the evaluation network to determine their value. This value score is compared with the previous score to determine whether further adjustments to the imaging mode are needed.

[0103] Through this network framework, the imaging system can intelligently adapt to different detection tasks and environmental conditions, automatically select the best imaging mode to improve image quality and detection accuracy. This intelligent decision-making algorithm has broad application prospects in the fields of automated monitoring, remote sensing detection, and reconnaissance.

[0104] Through this intelligent decision-making algorithm, the imaging system can automatically adjust its working mode in a changing environment to achieve the best detection effect. This algorithm has broad application prospects in reconnaissance, environmental monitoring, medical imaging and other fields, and can significantly improve the performance and adaptability of the imaging system.

[0105] Fig.10 A schematic diagram of online learning and offline learning of an intelligent decision-making algorithm network according to some embodiments of the present invention is shown.

[0106] like Fig.10 As shown, online learning and offline learning are two different training methods in reinforcement learning, and they can have different applications and characteristics in the evolution and training process of intelligent decision-making algorithms.

[0107] Online learning, also known as online reinforcement learning, refers to a method of updating a model in real time while interacting with the environment. In this method, the agent acquires new experience from the environment at each time step and immediately uses this experience to update its strategy or value function. The advantage of this method is that it can adapt to changes in the environment in real time, but the disadvantage is that it may take more time to converge because each update relies on the latest data.

[0108] Offline learning, also known as batch reinforcement learning, refers to the agent learning from a pre-collected dataset rather than interacting with the environment in real time. This approach allows the agent to train without environmental feedback, which means it can learn from historical data without the cost of real-time interaction. The advantages of offline learning include faster sampling efficiency and update frequency because it can process data in parallel. However, it also faces the drift problem of model trajectory learning, that is, the learned data may not match the current strategy, resulting in performance degradation. To solve this problem, an importance sampling coefficient can be introduced to correct the deviation of the data.

[0109] In the Actor-Critic reinforcement learning framework, the training process involves two neural networks: the actor network and the critic network. The actor network is responsible for generating policies, that is, the probability distribution of actions to choose in a given state; the critic network evaluates the value of these actions. The training process usually includes the following steps:

[0110] Randomly initialize the policy network (actor) and value network (critic).

[0111] Use the current policy to interact in the environment and collect state-action-reward pairs.

[0112] Update the value network using the collected data, which typically involves computing the temporal difference (TD) error.

[0113] The policy network is updated using the advantage function and gradient descent method to improve the quality of the policy.

[0114] Repeat the above process until the maximum number of iterations is reached or the convergence condition is met.

[0115] The advantage function plays a role in stabilizing the training process in the actor-critic framework. It measures the superiority of taking a certain action relative to the average situation. By optimizing the advantage function, the policy network can be updated more efficiently, thereby improving the performance of the agent.

[0116] In offline learning, experience replay allows the agent to learn from past experiences instead of relying solely on the latest interactions. This helps break temporal dependencies, improves data utilization, and reduces variance during training.

[0117] By combining the advantages of online and offline learning, and leveraging the actor-critic framework and experience replay technology, the intelligent decision-making algorithm is more efficient and stable.

[0118] Fig.11 A schematic block diagram of an unmanned autonomous aerial vehicle detection device according to an embodiment of the present invention is shown.

[0119] like Fig.11As shown, the unmanned autonomous aerial vehicle detection device includes a current detection information acquisition module 301, a target imaging mode selection module 302, and a target detection information output module 303. The current detection information acquisition module 301 is configured to obtain current detection information by detecting a current optical imaging mode of a group of multidimensional variable optical imaging modes, wherein the group of multidimensional variable optical imaging modes is associated with spatial information, spectral information, and polarization information; the target imaging mode selection module 302 is configured to select a target imaging mode from a group of multidimensional variable optical imaging modes based on the current detection information; and the target detection information output module 303 is configured to process the current detection information by using the target imaging mode to output the target detection information.

[0120] In some embodiments, the unmanned autonomous aerial vehicle detection device is further configured to obtain and select the spatial resolution of the target imaging mode using the first field of view switching module and the second field of view switching module; and determine the target imaging mode based on the spatial resolution.

[0121] In some embodiments, the unmanned autonomous aerial vehicle detection device is further configured to perform band and time fusion on current detection information to enhance the current detection information.

[0122] In some embodiments, the unmanned autonomous aerial vehicle detection device is further configured to use a target detection network to obtain at least one of current detection information and target detection information, wherein the target detection network includes an input terminal Input, a backbone network Backbone, a Neck network and a prediction terminal Prediction, wherein the input terminal Input includes Mosaic data enhancement, image size scaling and adaptive anchor frame calculation, and wherein the backbone network Backbone includes Focus processing, CBL processing, CSP structure and SPP spatial pyramid pooling.

[0123] In some embodiments, the unmanned autonomous aerial vehicle detection device is further configured to: input a set of parameter formatting of a multi-dimensional variable optical imaging mode; input current detection information; output parameter formatting of a target imaging mode; and obtain the target imaging mode by processing the parameter formatting output based on a reinforcement learning intelligent decision network.

[0124] In some embodiments, the intelligent decision network is trained in the following manner: randomly initializing the policy network and the value network; using the policy to interact in the environment to obtain state, action, and reward pairs; using the derived actor-critic network parameter update formula as training data to update the value network; using the gradient descent method, using the defined advantage function, and updating the policy network with the derived Actor network parameter update formula; and if the maximum number of iterations is reached, the training phase ends; if the maximum number of iterations is not reached, returning to the second step to continue interacting with the environment.

[0125] In one embodiment, the spatial information includes two-dimensional spatial information, and the polarization information includes imaging mode information in four polarization directions.

[0126] Fig.12 A block diagram of a computing device capable of implementing various embodiments of the present invention is shown. Fig.12 1 shows a schematic block diagram of an example device 1200 that can be used to implement an embodiment of the present invention. The device 1200 can be used to implement Figure 1 1. As shown in the figure, the device 1200 includes a CPU 1201 (central processing unit), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 1202 or loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An I / O interface 1205 (input / output interface) is also connected to the bus 1204.

[0127] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The CPU 1201 performs the various methods and processes described above, such as process 800 and / or process 900. For example, in some embodiments, process 800 and / or process 900 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by CPU 1201, one or more steps of process 800 and / or process 900 described above may be performed. Alternatively, in other embodiments, CPU 1201 may be configured to perform process 800 and / or process 900 in any other appropriate manner (e.g., by means of firmware).

[0129] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0130] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0133] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for detecting an unmanned autonomous aerial vehicle, characterized in that: include: Detecting current detection information using a current optical imaging mode of a set of multi-dimensional variable optical imaging modes, wherein the set of multi-dimensional variable optical imaging modes is associated with spatial information, spectral information, and polarization information; Based on the current detection information, selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes, the target imaging mode corresponding to specific spatial information, spectral information and polarization information; as well as The current detection information is processed using the target imaging mode to output target detection information.

2. The method according to claim 1, characterized in that Based on the current detection information, selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes comprises: Using the first field of view switching module and the second field of view switching module, obtaining a spatial resolution corresponding to the selected target imaging mode; and Based on the spatial resolution, the target imaging mode is determined.

3. The method according to claim 1, characterized in that Based on the current detection information, selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes comprises: The current detection information is subjected to band and time fusion to enhance the current detection information.

4. The method according to claim 3, characterized in that Also includes: A target detection network is used to obtain at least one of the current detection information and the target detection information, wherein the target detection network includes an input terminal Input, a backbone network Backbone, a Neck network and a prediction terminal Prediction, wherein the input terminal Input includes Mosaic data enhancement, image size scaling and adaptive anchor frame calculation, and wherein the backbone network Backbone includes Focus processing, CBL processing, CSP structure and SPP spatial pyramid pooling.

5. The method according to claim 4, characterized in that Based on the current detection information, selecting a target imaging mode from the set of multi-dimensional variable optical imaging modes comprises: Formatted input of parameters of the set of multi-dimensional variable optical imaging modes; Inputting the current detection information; Formatted output of parameters of the target imaging mode; and The target imaging mode is obtained by processing the parameter formatted output based on a reinforcement learning intelligent decision network.

6. The method according to claim 5, characterized in that The intelligent decision network is trained in the following way: Randomly initialize the policy network and value network; Use strategies to interact in the environment to obtain state, action, and reward pairs; Use the derived update formula of the actor-critic network parameters as training data to update the value network; Use the gradient descent method to update the policy network using the defined advantage function and the derived update formula for the Actor network parameters; If the maximum number of iterations is reached, the training phase ends; and If the maximum number of iterations is not reached, return to the second step to continue interacting with the environment.

7. The method according to claim 1, characterized in that The spatial information includes two-dimensional spatial information, and the polarization information includes imaging mode information in four polarization directions.

8. An unmanned autonomous aerial vehicle detection device, characterized in that: include: A current detection information acquisition module is configured to obtain current detection information by detecting a current optical imaging mode of a set of multi-dimensional variable optical imaging modes, wherein the set of multi-dimensional variable optical imaging modes is associated with spatial information, spectral information, and polarization information; a target imaging mode selection module, configured to select a target imaging mode from the set of multi-dimensional variable optical imaging modes based on the current detection information, the target imaging mode corresponding to the specific spatial information, the spectral information and the polarization information; as well as The target detection information output module is configured to process the current detection information using the target imaging mode to output target detection information.

9. An electronic device, comprising: one or more processors; as well as A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images